-- Logs begin at Tue 2022-07-26 12:59:01 UTC, end at Tue 2022-07-26 14:35:59 UTC. -- Jul 26 13:31:57.067492 dv-cinder-825492-30-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 26 13:31:57.076234 dv-cinder-825492-30-iscsi devstack@keystone.service[86925]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 26 13:31:57.078827 dv-cinder-825492-30-iscsi devstack@keystone.service[86925]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Jul 26 13:31:57 2022] *** Jul 26 13:31:57.078827 dv-cinder-825492-30-iscsi devstack@keystone.service[86925]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 26 13:31:57.078827 dv-cinder-825492-30-iscsi devstack@keystone.service[86925]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 26 13:31:57.078827 dv-cinder-825492-30-iscsi devstack@keystone.service[86925]: nodename: dv-cinder-825492-30-iscsi Jul 26 13:31:57.078827 dv-cinder-825492-30-iscsi devstack@keystone.service[86925]: machine: x86_64 Jul 26 13:31:57.078827 dv-cinder-825492-30-iscsi devstack@keystone.service[86925]: clock source: unix Jul 26 13:31:57.078827 dv-cinder-825492-30-iscsi devstack@keystone.service[86925]: pcre jit disabled Jul 26 13:31:57.078827 dv-cinder-825492-30-iscsi devstack@keystone.service[86925]: detected number of CPU cores: 4 Jul 26 13:31:57.078827 dv-cinder-825492-30-iscsi devstack@keystone.service[86925]: current working directory: / Jul 26 13:31:57.078827 dv-cinder-825492-30-iscsi devstack@keystone.service[86925]: detected binary path: /usr/bin/uwsgi-core Jul 26 13:31:57.078827 dv-cinder-825492-30-iscsi devstack@keystone.service[86925]: your processes number limit is 47680 Jul 26 13:31:57.079426 dv-cinder-825492-30-iscsi devstack@keystone.service[86925]: your memory page size is 4096 bytes Jul 26 13:31:57.079426 dv-cinder-825492-30-iscsi devstack@keystone.service[86925]: detected max file descriptor number: 2048 Jul 26 13:31:57.079426 dv-cinder-825492-30-iscsi devstack@keystone.service[86925]: lock engine: pthread robust mutexes Jul 26 13:31:57.079426 dv-cinder-825492-30-iscsi devstack@keystone.service[86925]: thunder lock: enabled Jul 26 13:31:57.079426 dv-cinder-825492-30-iscsi devstack@keystone.service[86925]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 26 13:31:57.079426 dv-cinder-825492-30-iscsi devstack@keystone.service[86925]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 26 13:31:57.117367 dv-cinder-825492-30-iscsi devstack@keystone.service[86925]: Python main interpreter initialized at 0x55f132a3c9c0 Jul 26 13:31:57.117367 dv-cinder-825492-30-iscsi devstack@keystone.service[86925]: python threads support enabled Jul 26 13:31:57.117367 dv-cinder-825492-30-iscsi devstack@keystone.service[86925]: your server socket listen backlog is limited to 100 connections Jul 26 13:31:57.117367 dv-cinder-825492-30-iscsi devstack@keystone.service[86925]: your mercy for graceful operations on workers is 90 seconds Jul 26 13:31:57.117715 dv-cinder-825492-30-iscsi devstack@keystone.service[86925]: mapped 403077 bytes (393 KB) for 2 cores Jul 26 13:31:57.117811 dv-cinder-825492-30-iscsi devstack@keystone.service[86925]: *** Operational MODE: preforking *** Jul 26 13:31:57.118082 dv-cinder-825492-30-iscsi devstack@keystone.service[86925]: *** uWSGI is running in multiple interpreter mode *** Jul 26 13:31:57.118082 dv-cinder-825492-30-iscsi devstack@keystone.service[86925]: spawned uWSGI master process (pid: 86925) Jul 26 13:31:57.118266 dv-cinder-825492-30-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 26 13:31:57.118773 dv-cinder-825492-30-iscsi devstack@keystone.service[86925]: spawned uWSGI worker 1 (pid: 86933, cores: 1) Jul 26 13:31:57.119088 dv-cinder-825492-30-iscsi devstack@keystone.service[86925]: spawned uWSGI worker 2 (pid: 86934, cores: 1) Jul 26 13:31:57.121041 dv-cinder-825492-30-iscsi devstack@keystone.service[86925]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 26 13:32:02.598829 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 26 13:32:02.598829 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: from cryptography.utils import int_from_bytes Jul 26 13:32:02.599655 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 26 13:32:02.599655 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: from cryptography.utils import int_from_bytes Jul 26 13:32:02.599655 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 26 13:32:02.599655 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: from cryptography.utils import int_from_bytes Jul 26 13:32:02.600166 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 26 13:32:02.600166 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: from cryptography.utils import int_from_bytes Jul 26 13:32:02.649093 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 26 13:32:02.649314 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 26 13:32:02.649404 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 26 13:32:02.649526 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] command line args: [] {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 26 13:32:02.649611 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 26 13:32:02.649690 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] command line args: [] {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 26 13:32:02.649768 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 26 13:32:02.649904 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] ================================================================================ {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 26 13:32:02.649990 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 26 13:32:02.650124 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ================================================================================ {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 26 13:32:02.650202 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] admin_token = **** {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.650381 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] admin_token = **** {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.650468 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] config_dir = [] {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.650565 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.650648 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] config_dir = [] {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.650748 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] config_source = [] {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.650832 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.650929 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] debug = True {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.651018 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] config_source = [] {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.651128 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.651386 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] debug = True {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.651386 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.651531 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.651598 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.651660 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] insecure_debug = False {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.651722 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] insecure_debug = False {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.651908 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.652037 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.652161 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.652265 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.652392 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] list_limit = None {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.652500 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] list_limit = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.652606 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] log_config_append = None {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.652709 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] log_config_append = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.652804 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.652902 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.653001 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] log_dir = None {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.653101 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] log_dir = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.653199 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] log_file = None {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.653300 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] log_file = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.653411 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.653524 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.653655 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.653731 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.653849 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.653931 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.654008 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.654131 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.654211 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.654326 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.654412 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.654491 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.654615 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.654690 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.654803 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.654876 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.655001 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.655081 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.655232 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.655305 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.655468 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.656165 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.656259 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.656338 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.656495 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.656622 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.656753 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] notification_format = cadf {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.656908 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] notification_format = cadf {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.657050 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.657125 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.657528 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.58/identity {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.657684 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.58/identity {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.657772 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] publish_errors = False {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.657844 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] publish_errors = False {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.657930 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.658048 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.658161 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.658244 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.658340 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.658421 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.658582 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.658715 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.658819 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.658896 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.659051 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] standard_threads = False {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.659130 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] standard_threads = False {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.659281 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] strict_password_check = False {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.659411 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] strict_password_check = False {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.659522 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.659598 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.659699 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] use_eventlog = False {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.659775 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] use_eventlog = False {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.659872 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] use_journal = False {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.659954 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] use_journal = False {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.660032 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] use_json = False {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.660101 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] use_json = False {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.660297 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] use_stderr = False {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.660388 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] use_stderr = False {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.660388 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] use_syslog = False {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.660533 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] use_syslog = False {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.660610 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] watch_log_file = False {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.660782 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] watch_log_file = False {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 13:32:02.660856 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.660931 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.661065 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.661201 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.661326 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.661410 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.661591 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.661668 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.661841 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.661920 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.662083 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.662170 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.662328 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.662405 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.662564 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.662648 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.662810 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.662888 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.663056 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.663133 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.663305 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.663382 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.663538 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.663614 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.663784 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.663861 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.664018 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.664100 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.664264 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.664367 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.664509 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.664585 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.664738 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.664817 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.664986 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.665061 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.665236 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.665313 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.665563 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.665644 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.665734 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.665810 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.665988 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.666063 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.666245 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.666322 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.666496 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.666574 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.666745 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.666820 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.666976 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.667058 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.667207 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.667283 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.667436 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.667552 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.667656 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.667753 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.667904 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.667987 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.668150 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.668237 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.668380 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.668461 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.668608 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.668683 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.668853 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.668940 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.669100 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.669175 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.669334 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.669398 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.669603 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.669685 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.669834 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.669898 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.670070 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.670134 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.670304 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.670448 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.670535 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.670600 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.670763 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.670827 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.670993 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.671069 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.671250 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.671315 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.671487 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.671551 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.671736 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.671812 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.671966 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.672032 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.672207 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.672272 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.672435 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.672511 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.672671 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.672736 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.672905 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.672970 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.673150 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.673226 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.673379 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.673453 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.673641 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.673708 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.673879 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.673958 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.674115 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.674182 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.674354 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.674419 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.674588 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.674661 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.674819 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.674885 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.675066 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.675132 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.675302 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.675376 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.675538 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.675606 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.675769 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.675850 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.676354 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.676438 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.676498 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.676564 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.676624 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.676680 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.676736 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.676793 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.676931 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.677008 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.677445 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.677572 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.677711 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.677779 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.677958 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.678033 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.678202 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.678268 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.678438 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.678504 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.678673 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.678741 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.678925 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.679391 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.679520 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] cache.enabled = True {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.679624 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] cache.enabled = True {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.679733 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.679834 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.679965 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.680069 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.680168 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.680267 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.680373 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.680467 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.680537 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.680595 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.680652 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.680709 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.680774 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.680833 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.680945 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.681024 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.681208 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.681297 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.681448 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.681639 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.681737 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.681832 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.681927 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.682021 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.682197 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.682305 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.682409 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.682504 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.682619 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.682724 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.682854 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.682969 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.683115 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.683244 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.683364 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.683488 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.683608 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.683736 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.683821 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.683882 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.683974 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.684052 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.684215 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.684313 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.684445 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.684527 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.684684 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.684762 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.684920 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.684997 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.685175 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.685253 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.685432 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.685543 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.685695 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.685831 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.685919 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] auth.external = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.685991 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] auth.external = None {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.686152 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] auth.mapped = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.686279 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] auth.mapped = None {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.686452 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.686607 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.686742 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.686876 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.687005 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] auth.password = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.687130 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] auth.password = None {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.687254 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] auth.token = None {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.687378 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] auth.token = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.687502 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.687646 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.687778 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] catalog.caching = True {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.687897 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] catalog.caching = True {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.688033 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.688159 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.688291 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.688424 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.688550 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.688676 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.688801 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.688926 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.689052 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.689185 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.689320 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] credential.caching = True {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.689445 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] credential.caching = True {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.689618 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] credential.driver = sql {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.689744 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] credential.driver = sql {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.689878 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.690012 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.690147 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.690272 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.690397 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.690519 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.690594 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.690674 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.690745 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.690815 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.690885 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.690954 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.691023 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.691100 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.691172 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.691243 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.691341 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.691427 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.691630 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.691719 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.691879 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.691948 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.692575 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.692678 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.692752 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.692844 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.692918 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.692989 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.693060 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] federation.caching = True {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.693136 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] federation.caching = True {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.693287 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.693381 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.693456 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] federation.driver = sql {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.693558 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] federation.driver = sql {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.693630 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.693794 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.693873 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.694029 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.694110 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.694265 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.694344 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.694510 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.694588 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.694747 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.694826 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.694994 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.695071 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.695230 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.695307 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.695483 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.695563 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.695717 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] identity.caching = True {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.695794 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.695965 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.696044 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] identity.caching = True {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.696211 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.696292 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.696448 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.696527 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.696685 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.696771 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.696924 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.697003 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] identity.driver = sql {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.697156 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.697233 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] identity.driver = sql {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.697396 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.697502 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.697673 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.697750 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.697928 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.698006 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.698184 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.698274 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.698417 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.698496 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.698651 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.698729 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.698888 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.698974 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.699133 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.699210 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.699380 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.699458 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.699628 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.699713 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.699876 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.700000 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.700126 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.700208 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.700371 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.700463 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.700610 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.700690 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.700841 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.700920 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.701080 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.701167 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.701308 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.701387 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.701564 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.701645 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.701806 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.701911 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.702046 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.702126 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.702289 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.702878 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.703047 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.703137 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.703288 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.703367 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.703533 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.703613 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.703771 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.703868 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.704005 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.704084 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.704251 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.704329 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.704484 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.704572 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.704719 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.704798 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.704948 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] ldap.password = **** {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.705026 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.705184 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.password = **** {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.705271 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.705420 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.705512 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.705686 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.705765 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.705924 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.706017 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.706162 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.706241 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.706402 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.706487 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.706647 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.706734 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.706879 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.706959 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.707110 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.707191 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.707343 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.707430 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.707592 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.707671 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.707828 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.707908 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.708064 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.708164 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.708299 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.708380 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.708530 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] ldap.user = None {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.708610 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.708763 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.user = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.708850 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.709006 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.709086 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.709236 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.709315 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.709498 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.709583 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.709748 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.709829 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.709996 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.710075 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.710231 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.710318 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.710459 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.710539 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.710690 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.710768 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.710922 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.711003 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.711159 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.711238 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.711389 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.711468 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.711621 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.711701 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.711869 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.711949 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.712116 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.712196 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.712364 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.712507 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.712753 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.712832 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.712981 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.713059 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.713227 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.713307 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.713463 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.713560 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.713728 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.713808 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.713965 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.714047 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.714201 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.714281 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.714438 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.714517 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.714682 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.714763 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.714920 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.714999 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.715158 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.715238 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] policy.driver = sql {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.715390 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.715475 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] policy.driver = sql {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.715634 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.715713 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.715870 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.715953 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.716105 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] receipt.caching = True {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.716186 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.716340 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] receipt.caching = True {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.716418 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.716580 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.716659 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.716819 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.716902 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.717051 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.717130 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.717281 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.717360 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.717539 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] resource.caching = True {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.717623 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.717781 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] resource.caching = True {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.717860 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.718079 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] resource.driver = sql {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.718680 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.718815 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.718897 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] resource.driver = sql {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.719057 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.719142 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.719307 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.719385 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.719544 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] revoke.caching = True {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.719626 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.719786 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] revoke.caching = True {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.719864 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.720038 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.720170 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.720380 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] role.cache_time = None {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.720512 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.720640 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] role.caching = True {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.720765 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] role.cache_time = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.720765 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] role.caching = True {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.721006 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] role.driver = sql {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.721006 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] role.list_limit = None {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.721248 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] role.driver = sql {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.721248 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] role.list_limit = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.721506 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.721506 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.721759 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.721898 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.722024 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.722151 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.722279 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.722405 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.722541 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.722671 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.722797 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.722924 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.723056 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.723183 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.723328 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.723457 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.723584 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.723712 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.723838 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.723961 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.724097 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.724223 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.724305 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.724434 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.724560 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.724560 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.724802 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.724802 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.725055 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.725182 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.725309 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.725435 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.725552 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.725672 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.725797 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.725921 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.726045 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.726168 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.726307 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.726432 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.726556 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.726690 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.726816 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.726940 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.727071 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.727196 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.727319 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.727442 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.727566 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.727698 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.727831 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.727957 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.728081 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.728215 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.728357 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.728490 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.728616 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.728740 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.728865 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.728989 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.729111 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.729242 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] token.cache_time = None {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.729367 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.729561 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] token.caching = True {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.729686 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] token.cache_time = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.729814 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.729942 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] token.caching = True {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.730079 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] token.provider = fernet {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.730221 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.730349 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.730476 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] token.provider = fernet {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.730602 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.730729 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.730863 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.730991 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.731118 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.731244 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.731370 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.731496 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.731630 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.731756 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.731892 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] trust.driver = sql {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.732020 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.732147 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.732294 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] trust.driver = sql {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.732439 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.732519 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.732581 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.732640 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.732699 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.732757 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.732836 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.732911 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.732982 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.733052 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.733124 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.733195 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.733275 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.733349 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.733420 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] database.connection = **** {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.733521 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.733605 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.733668 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] database.connection = **** {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.733748 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.733829 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.733910 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.733991 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.734064 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.734183 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.734264 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.734421 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.734500 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.734659 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.734739 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.734893 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.734971 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.735131 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.735218 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.735365 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.735445 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.735609 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.735689 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.735844 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.735924 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.736088 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.736168 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.736324 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.736403 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.736571 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.736650 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.736801 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.736883 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.737667 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.737771 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.737771 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.737895 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.737895 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86933) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 26 13:32:02.738047 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.738047 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 13:32:02.738047 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 26 13:32:02.767308 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=86933) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 13:32:02.767536 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=86933) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 13:32:02.767763 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=86933) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 13:32:02.769146 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=86934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 13:32:02.769379 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=86934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 13:32:02.769626 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=86934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 13:32:02.776170 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=86933) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 13:32:02.778081 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=86934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 13:32:02.806797 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=86933) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 13:32:02.807027 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=86933) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 13:32:02.808998 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=86934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 13:32:02.809234 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=86934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 13:32:02.861889 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=86934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 13:32:02.862515 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=86933) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 13:32:02.913807 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=86933) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 13:32:02.914049 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=86933) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 13:32:02.914611 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=86934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 13:32:02.914854 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=86934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 13:32:02.984794 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86933) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 13:32:02.985021 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=86933) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 13:32:02.985149 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 13:32:02.985237 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=86933) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 13:32:02.985384 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=86934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 13:32:02.985486 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=86933) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 13:32:02.985634 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=86934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 13:32:02.985716 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=86933) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 13:32:02.985858 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=86934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 13:32:02.985938 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=86933) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 13:32:02.986076 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=86934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 13:32:02.986292 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=86934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 13:32:03.009218 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86933) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 13:32:03.009451 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=86933) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 13:32:03.009657 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 13:32:03.009773 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=86933) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 13:32:03.009891 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=86933) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 13:32:03.009975 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=86934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 13:32:03.010104 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=86933) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 13:32:03.010187 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=86934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 13:32:03.010316 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=86933) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 13:32:03.010400 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=86934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 13:32:03.010528 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86933) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 13:32:03.010611 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=86934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 13:32:03.010742 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=86934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 13:32:03.010958 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 13:32:03.022214 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=86933) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 13:32:03.022466 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=86934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 13:32:03.035320 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86933) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 13:32:03.035544 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=86933) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 13:32:03.035633 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 13:32:03.035765 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=86933) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 13:32:03.035849 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=86934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 13:32:03.035985 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=86933) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 13:32:03.036068 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=86934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 13:32:03.036210 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=86934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 13:32:03.036302 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=86933) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 13:32:03.036417 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=86933) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 13:32:03.036497 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=86934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 13:32:03.036631 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86933) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 13:32:03.036724 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=86934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 13:32:03.036877 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 13:32:03.053989 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=86933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.054197 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=86933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 26 13:32:03.054532 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=86933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.054715 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.054823 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=86933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 26 13:32:03.054928 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 26 13:32:03.055013 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=86933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.055218 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=86933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.055300 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.055449 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 26 13:32:03.055552 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=86933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.055716 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=86933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.055797 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.055972 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.056227 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.056482 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.065821 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=86933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.066095 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=86933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.066361 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=86933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.066639 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=86933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.066723 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.066926 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.067194 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.067471 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.073441 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=86933) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 13:32:03.074329 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=86934) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 13:32:03.078725 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=86933) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 13:32:03.079005 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=86933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.079286 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=86933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.079566 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=86933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.079648 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=86934) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 13:32:03.079885 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=86933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.079969 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.080163 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=86933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.080245 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.080427 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=86933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.080518 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.080705 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=86933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.080787 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.080962 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.081042 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=86933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.081232 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.081323 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=86933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.081534 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=86933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.081789 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.082067 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.082355 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.082634 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.104313 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=86933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.105523 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.105829 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=86933) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 13:32:03.106114 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=86933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.107003 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=86934) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 13:32:03.107293 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.110763 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=86933) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 13:32:03.111041 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=86933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.111321 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=86933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.111996 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=86934) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 13:32:03.112232 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.112514 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.117921 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=86933) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 13:32:03.118201 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=86933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.119139 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=86934) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 13:32:03.119421 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.122296 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=86933) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 13:32:03.122581 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=86933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.122871 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=86933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.123153 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=86933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.123426 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=86933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.123708 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=86933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.123789 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=86934) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 13:32:03.123976 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=86933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.124059 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.124768 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=86933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.124870 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.125066 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.125341 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.125649 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.125927 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.126201 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.141324 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=86933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.141633 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=86933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.142480 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.142761 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.145503 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=86933) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 13:32:03.146671 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=86934) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 13:32:03.148616 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=86933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.148911 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=86933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.149795 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.150091 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.153239 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=86933) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 13:32:03.154457 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=86934) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 13:32:03.156456 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=86933) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 13:32:03.157657 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=86934) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 13:32:03.159623 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=86933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.159914 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=86933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.160196 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=86933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.160485 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=86933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.160768 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=86933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.160850 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.161060 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=86933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.161143 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.161405 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.161716 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.161998 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.162280 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.176799 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=86933) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 13:32:03.177084 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=86933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.177772 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=86933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.178000 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=86933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.178919 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=86934) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 13:32:03.179207 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.179474 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.179737 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.184890 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=86933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.186457 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=86933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.186603 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.187274 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=86933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.188663 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.188938 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.190067 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=86933) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 13:32:03.190347 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=86933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.190645 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=86933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.190927 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=86933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.191212 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=86933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.191762 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=86934) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 13:32:03.192038 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.192342 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.192618 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.192909 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.203347 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=86933) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 13:32:03.203621 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=86933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.203897 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=86933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.204172 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=86933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.204446 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=86933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.204715 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=86933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.204916 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=86934) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 13:32:03.205000 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=86933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.205198 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.205502 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.205785 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.206067 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.206341 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.206624 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.220382 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=86933) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 13:32:03.222084 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=86934) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 13:32:03.223323 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=86933) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 13:32:03.225061 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=86934) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 13:32:03.226259 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=86933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.228004 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.228830 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=86933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.230345 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=86933) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 13:32:03.230627 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.230710 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=86933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.230929 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=86933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.232124 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=86934) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 13:32:03.232409 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.232692 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.237329 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=86933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.238938 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=86933) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 13:32:03.239139 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.240715 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=86934) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 13:32:03.241852 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=86933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.242139 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=86933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.242419 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=86933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.242692 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=86933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.243710 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.244000 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.244286 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.244565 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.251258 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=86933) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 13:32:03.251546 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=86933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.251826 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=86933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.252992 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=86934) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 13:32:03.253282 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.253594 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.258605 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=86933) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 13:32:03.258882 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=86933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.259159 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=86933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.259442 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=86933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.259711 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=86933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.259995 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=86933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.260263 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=86933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.260386 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=86934) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 13:32:03.260532 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=86933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.260665 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.260909 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=86933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.260990 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.261190 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=86933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.261279 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.261494 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.261577 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=86933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.261754 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=86933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.261892 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.262029 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=86933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.262109 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.262298 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=86933) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.262427 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.262667 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.262950 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.263237 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.263512 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.263789 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.264063 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 13:32:03.295603 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55f132a3c9c0 pid: 86933 (default app) Jul 26 13:32:03.298738 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55f132a3c9c0 pid: 86934 (default app) Jul 26 13:32:03.299557 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc1ccfa-fbf0-437e-a071-562405aed68c None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:03.299845 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc1ccfa-fbf0-437e-a071-562405aed68c None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:03.300118 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc1ccfa-fbf0-437e-a071-562405aed68c None None] PATH_INFO: `/v3` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:03.304700 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1/1] 10.222.0.58 () {50 vars in 791 bytes} [Tue Jul 26 13:32:03 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:10.993160 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a6ae4b-7220-43fb-bc1e-56b75c196f2f None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:10.993756 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a6ae4b-7220-43fb-bc1e-56b75c196f2f None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:10.993756 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a6ae4b-7220-43fb-bc1e-56b75c196f2f None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:10.998134 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1/2] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:32:10 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 13:32:11.003635 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d58371-66b8-4262-901c-93ac331febcd None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:11.004084 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d58371-66b8-4262-901c-93ac331febcd None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:11.004175 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d58371-66b8-4262-901c-93ac331febcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:11.075233 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: INFO dbcounter [None req-97d58371-66b8-4262-901c-93ac331febcd None None] Registered counter for database keystone Jul 26 13:32:11.097766 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG oslo_db.sqlalchemy.engines [None req-97d58371-66b8-4262-901c-93ac331febcd None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=86933) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 26 13:32:11.104157 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writer thread running {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 26 13:32:11.290269 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG passlib.handlers.bcrypt [None req-97d58371-66b8-4262-901c-93ac331febcd None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=86933) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 26 13:32:11.290609 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG passlib.handlers.bcrypt [None req-97d58371-66b8-4262-901c-93ac331febcd None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=86933) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 26 13:32:11.346497 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-97d58371-66b8-4262-901c-93ac331febcd None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:32:11.348134 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-97d58371-66b8-4262-901c-93ac331febcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:32:11.472554 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 2/3] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:32:11 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 470 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 26 13:32:11.603124 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a6ae4b-7220-43fb-bc1e-56b75c196f2f None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:11.673183 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: INFO dbcounter [None req-73a6ae4b-7220-43fb-bc1e-56b75c196f2f None None] Registered counter for database keystone Jul 26 13:32:11.693335 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG oslo_db.sqlalchemy.engines [None req-73a6ae4b-7220-43fb-bc1e-56b75c196f2f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=86934) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 26 13:32:11.699505 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writer thread running {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 26 13:32:11.818868 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a6ae4b-7220-43fb-bc1e-56b75c196f2f None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:11.823156 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d73c44-5ea5-4295-8f3d-da800ba51f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:11.823794 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d73c44-5ea5-4295-8f3d-da800ba51f5c admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:11.824078 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d73c44-5ea5-4295-8f3d-da800ba51f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:11.824355 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d73c44-5ea5-4295-8f3d-da800ba51f5c admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:11.832697 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d73c44-5ea5-4295-8f3d-da800ba51f5c admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:11.884203 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: WARNING keystone.common.rbac_enforcer.enforcer [None req-e3d73c44-5ea5-4295-8f3d-da800ba51f5c admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 26 13:32:11.884472 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d73c44-5ea5-4295-8f3d-da800ba51f5c admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:11.894963 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 2/4] 10.222.0.58 () {58 vars in 1324 bytes} [Tue Jul 26 13:32:11 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 26 13:32:11.900954 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d58371-66b8-4262-901c-93ac331febcd None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:11.915436 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d58371-66b8-4262-901c-93ac331febcd None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:11.919916 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c8a898-2689-481b-bbef-09715ffd755c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:11.921554 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c8a898-2689-481b-bbef-09715ffd755c admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:11.921844 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c8a898-2689-481b-bbef-09715ffd755c admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:11.922129 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c8a898-2689-481b-bbef-09715ffd755c admin admin] PATH_INFO: `/v3/projects/2e341f5b1be34081bfdda39547d48a7c` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:11.923613 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c8a898-2689-481b-bbef-09715ffd755c admin admin] RBAC: Authorizing `identity:get_project(project_id=2e341f5b1be34081bfdda39547d48a7c)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:11.974331 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: WARNING keystone.common.rbac_enforcer.enforcer [None req-f6c8a898-2689-481b-bbef-09715ffd755c admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 26 13:32:11.974677 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c8a898-2689-481b-bbef-09715ffd755c admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:11.979474 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 3/5] 10.222.0.58 () {56 vars in 1207 bytes} [Tue Jul 26 13:32:11 2022] GET /identity/v3/projects/2e341f5b1be34081bfdda39547d48a7c => generated 344 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:13.949719 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1300d94f-dbd3-467b-925d-099de560def7 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:13.950255 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1300d94f-dbd3-467b-925d-099de560def7 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:13.950255 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1300d94f-dbd3-467b-925d-099de560def7 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:13.954892 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 3/6] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:32:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 13:32:13.960346 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e32deb-e8dd-4610-ac40-2e7388746d12 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:13.960732 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e32deb-e8dd-4610-ac40-2e7388746d12 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:13.960905 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e32deb-e8dd-4610-ac40-2e7388746d12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:13.981206 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-63e32deb-e8dd-4610-ac40-2e7388746d12 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:32:13.982765 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-63e32deb-e8dd-4610-ac40-2e7388746d12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:32:13.990689 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 4/7] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:32:13 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 26 13:32:14.012084 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1300d94f-dbd3-467b-925d-099de560def7 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:14.024555 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1300d94f-dbd3-467b-925d-099de560def7 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:14.028717 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a7b0fc-034d-447b-87a5-4a0d83cc18ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:14.029321 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a7b0fc-034d-447b-87a5-4a0d83cc18ab admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:14.029643 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a7b0fc-034d-447b-87a5-4a0d83cc18ab admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:14.030500 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a7b0fc-034d-447b-87a5-4a0d83cc18ab admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:14.038547 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a7b0fc-034d-447b-87a5-4a0d83cc18ab admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:14.040389 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a7b0fc-034d-447b-87a5-4a0d83cc18ab admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:14.049221 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 4/8] 10.222.0.58 () {58 vars in 1324 bytes} [Tue Jul 26 13:32:14 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 13:32:14.055043 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e32deb-e8dd-4610-ac40-2e7388746d12 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:14.064000 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e32deb-e8dd-4610-ac40-2e7388746d12 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:14.068379 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5587c4-9b08-49ab-b21f-56a4524b0912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:14.069014 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5587c4-9b08-49ab-b21f-56a4524b0912 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:14.069298 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5587c4-9b08-49ab-b21f-56a4524b0912 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:14.069608 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5587c4-9b08-49ab-b21f-56a4524b0912 admin admin] PATH_INFO: `/v3/users/f1ff1f65366f4d20a56500bb6bc88c54` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:14.070965 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5587c4-9b08-49ab-b21f-56a4524b0912 admin admin] RBAC: Authorizing `identity:get_user(user_id=f1ff1f65366f4d20a56500bb6bc88c54)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:14.072779 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5587c4-9b08-49ab-b21f-56a4524b0912 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:14.074370 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 5/9] 10.222.0.58 () {56 vars in 1198 bytes} [Tue Jul 26 13:32:14 2022] GET /identity/v3/users/f1ff1f65366f4d20a56500bb6bc88c54 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:16.115174 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e6b5fb-3904-4c8c-acb8-10571d8a4331 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:16.115702 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e6b5fb-3904-4c8c-acb8-10571d8a4331 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:16.115702 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e6b5fb-3904-4c8c-acb8-10571d8a4331 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:16.117045 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 5/10] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:32:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:32:16.122196 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e08790-7000-4d24-8ffc-5d6a2feeb40d None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:16.122616 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e08790-7000-4d24-8ffc-5d6a2feeb40d None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:16.122748 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e08790-7000-4d24-8ffc-5d6a2feeb40d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:16.142433 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-61e08790-7000-4d24-8ffc-5d6a2feeb40d None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:32:16.143627 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-61e08790-7000-4d24-8ffc-5d6a2feeb40d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:32:16.170179 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2e14a6-5d80-49ee-bfac-7a35d8f20818 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:16.170959 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2e14a6-5d80-49ee-bfac-7a35d8f20818 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:16.171238 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2e14a6-5d80-49ee-bfac-7a35d8f20818 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:16.172782 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 6/11] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:32:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 13:32:16.178679 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7b9cb9-6d19-4434-b871-5d912985eeb6 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:16.178967 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7b9cb9-6d19-4434-b871-5d912985eeb6 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:16.179248 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7b9cb9-6d19-4434-b871-5d912985eeb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:16.221352 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG passlib.handlers.bcrypt [None req-5f7b9cb9-6d19-4434-b871-5d912985eeb6 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=86934) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 26 13:32:16.221802 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG passlib.handlers.bcrypt [None req-5f7b9cb9-6d19-4434-b871-5d912985eeb6 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=86934) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 26 13:32:16.226315 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 6/12] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:32:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 13:32:16.249968 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e08790-7000-4d24-8ffc-5d6a2feeb40d None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:16.268061 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-5f7b9cb9-6d19-4434-b871-5d912985eeb6 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:32:16.269683 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-5f7b9cb9-6d19-4434-b871-5d912985eeb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:32:16.272497 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:16.353113 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e08790-7000-4d24-8ffc-5d6a2feeb40d None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:16.370481 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 7/13] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:32:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 13:32:16.388187 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96eb2f1-ce3a-415a-bf5a-3fad88c0cc2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:16.388842 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96eb2f1-ce3a-415a-bf5a-3fad88c0cc2a None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:16.389152 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96eb2f1-ce3a-415a-bf5a-3fad88c0cc2a None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:16.389427 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96eb2f1-ce3a-415a-bf5a-3fad88c0cc2a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:16.392712 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7b9cb9-6d19-4434-b871-5d912985eeb6 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:16.401914 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96eb2f1-ce3a-415a-bf5a-3fad88c0cc2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:32:16.411162 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-c96eb2f1-ce3a-415a-bf5a-3fad88c0cc2a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 13:32:16.411162 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:32:16.411162 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:32:16.411162 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:32:16.411162 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:32:16.411162 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:32:16.411162 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:32:16.411162 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:32:16.411162 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:32:16.411162 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:32:16.411162 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:32:16.411162 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:32:16.411162 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:32:16.411162 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:32:16.411162 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:32:16.411162 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:32:16.412185 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:32:16.412185 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:32:16.412185 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:32:16.412185 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:32:16.412185 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:32:16.412185 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:32:16.412185 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:32:16.412185 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:32:16.412185 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:32:16.412185 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:32:16.412185 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:32:16.412185 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:32:16.412185 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:32:16.412185 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:32:16.412185 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:32:16.412185 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:32:16.413048 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:32:16.413048 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:32:16.413048 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:32:16.413048 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:32:16.413048 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:32:16.413048 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:32:16.413048 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:32:16.413048 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:32:16.413048 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:32:16.413048 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:32:16.413048 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 13:32:16.413048 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:32:16.413048 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 7/14] 10.222.0.58 () {56 vars in 1096 bytes} [Tue Jul 26 13:32:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 13:32:16.418150 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96eb2f1-ce3a-415a-bf5a-3fad88c0cc2a None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:16.493485 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7b9cb9-6d19-4434-b871-5d912985eeb6 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:16.495848 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:16.499982 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:16.519316 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96eb2f1-ce3a-415a-bf5a-3fad88c0cc2a None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:16.537462 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159dfedf-a882-48ae-9656-b5a03cd398e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:16.538375 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159dfedf-a882-48ae-9656-b5a03cd398e8 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:16.538669 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159dfedf-a882-48ae-9656-b5a03cd398e8 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:16.538941 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159dfedf-a882-48ae-9656-b5a03cd398e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:16.556069 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5a75b3-8cd5-4e9d-9f36-a270d2555607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:16.556703 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5a75b3-8cd5-4e9d-9f36-a270d2555607 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:16.556989 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5a75b3-8cd5-4e9d-9f36-a270d2555607 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:16.557266 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5a75b3-8cd5-4e9d-9f36-a270d2555607 None admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:16.558026 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5a75b3-8cd5-4e9d-9f36-a270d2555607 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:32:16.558334 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5a75b3-8cd5-4e9d-9f36-a270d2555607 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:16.559739 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5a75b3-8cd5-4e9d-9f36-a270d2555607 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:16.571822 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 8/15] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Jul 26 13:32:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:32:16.578044 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5a75b3-8cd5-4e9d-9f36-a270d2555607 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:16.611559 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159dfedf-a882-48ae-9656-b5a03cd398e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:16.613003 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159dfedf-a882-48ae-9656-b5a03cd398e8 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:16.685247 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5a75b3-8cd5-4e9d-9f36-a270d2555607 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:16.724199 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 8/16] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Jul 26 13:32:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 26 13:32:16.730438 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159dfedf-a882-48ae-9656-b5a03cd398e8 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:16.734576 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:16.828270 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f38f1b2-ed3b-4e74-a99e-fa1bc2600b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:16.828933 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f38f1b2-ed3b-4e74-a99e-fa1bc2600b5d None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:16.829218 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f38f1b2-ed3b-4e74-a99e-fa1bc2600b5d None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:16.829524 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f38f1b2-ed3b-4e74-a99e-fa1bc2600b5d None admin] PATH_INFO: `/v3/users/f1ff1f65366f4d20a56500bb6bc88c54` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:16.830796 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f38f1b2-ed3b-4e74-a99e-fa1bc2600b5d None admin] RBAC: Authorizing `identity:get_user(user_id=f1ff1f65366f4d20a56500bb6bc88c54)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:16.832304 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f38f1b2-ed3b-4e74-a99e-fa1bc2600b5d None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:16.833976 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 9/17] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Jul 26 13:32:16 2022] GET /identity/v3/users/f1ff1f65366f4d20a56500bb6bc88c54 => generated 247 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:32:16.839291 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f38f1b2-ed3b-4e74-a99e-fa1bc2600b5d None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:16.842109 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159dfedf-a882-48ae-9656-b5a03cd398e8 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:16.854976 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:16.880134 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b52fd2-f406-4f57-9008-63fe9eb21b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:16.880901 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b52fd2-f406-4f57-9008-63fe9eb21b89 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:16.881183 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b52fd2-f406-4f57-9008-63fe9eb21b89 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:16.881453 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b52fd2-f406-4f57-9008-63fe9eb21b89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:16.900226 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b52fd2-f406-4f57-9008-63fe9eb21b89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:16.901738 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b52fd2-f406-4f57-9008-63fe9eb21b89 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:16.920812 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-03b52fd2-f406-4f57-9008-63fe9eb21b89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:32:16.920812 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:32:16.920812 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:32:16.920812 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:32:16.920812 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:32:16.920812 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:32:16.920812 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:32:16.920812 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:32:16.920812 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:32:16.920812 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:32:16.920812 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:32:16.920812 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:32:16.920812 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 13:32:16.920812 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 13:32:16.920812 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 13:32:16.921761 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 13:32:16.921761 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:32:16.921761 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:32:16.921761 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:32:16.921761 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:32:16.921761 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:32:16.921761 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:32:16.921761 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:32:16.921761 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:32:16.921761 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:32:16.921761 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:32:16.921761 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:32:16.921761 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:32:16.921761 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:32:16.921761 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:32:16.921761 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:32:16.921761 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:32:16.922560 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 13:32:16.922560 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 13:32:16.922560 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:32:16.922560 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:32:16.922560 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 9/18] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:32:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 13:32:16.928207 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b52fd2-f406-4f57-9008-63fe9eb21b89 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:16.956325 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f38f1b2-ed3b-4e74-a99e-fa1bc2600b5d None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:16.991184 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d2cfec-1d90-48aa-925e-9dd72159bf18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:16.991395 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:16.991856 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d2cfec-1d90-48aa-925e-9dd72159bf18 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:16.992242 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d2cfec-1d90-48aa-925e-9dd72159bf18 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:16.992650 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d2cfec-1d90-48aa-925e-9dd72159bf18 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:16.994205 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d2cfec-1d90-48aa-925e-9dd72159bf18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:16.996222 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d2cfec-1d90-48aa-925e-9dd72159bf18 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:16.997943 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 10/19] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:32:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:32:17.003608 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d2cfec-1d90-48aa-925e-9dd72159bf18 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:17.036468 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b52fd2-f406-4f57-9008-63fe9eb21b89 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:17.071466 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d5b145-9e8d-47cd-9b6d-e80a12000950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:17.072084 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d5b145-9e8d-47cd-9b6d-e80a12000950 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:17.072363 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d5b145-9e8d-47cd-9b6d-e80a12000950 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:17.072628 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d5b145-9e8d-47cd-9b6d-e80a12000950 None admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:17.073323 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d5b145-9e8d-47cd-9b6d-e80a12000950 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:32:17.073671 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d5b145-9e8d-47cd-9b6d-e80a12000950 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:17.075134 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d5b145-9e8d-47cd-9b6d-e80a12000950 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:17.093686 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:17.096868 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 10/20] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Jul 26 13:32:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:32:17.115859 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d2cfec-1d90-48aa-925e-9dd72159bf18 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:17.153954 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01332eec-de8e-4620-b689-ad84cd9f4e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:17.154640 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01332eec-de8e-4620-b689-ad84cd9f4e7b None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:17.155361 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01332eec-de8e-4620-b689-ad84cd9f4e7b None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:17.155621 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01332eec-de8e-4620-b689-ad84cd9f4e7b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:17.156451 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01332eec-de8e-4620-b689-ad84cd9f4e7b None admin] RBAC: Adding query filter params (role.id=6e0201a9a6484f7d8b112b1eafff70bd, scope.domain.id=default, user.id=f1ff1f65366f4d20a56500bb6bc88c54) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:32:17.156753 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01332eec-de8e-4620-b689-ad84cd9f4e7b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:17.158246 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01332eec-de8e-4620-b689-ad84cd9f4e7b None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:17.165916 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 11/21] 10.222.0.58 () {56 vars in 1322 bytes} [Tue Jul 26 13:32:17 2022] GET /identity/v3/role_assignments?user.id=f1ff1f65366f4d20a56500bb6bc88c54&scope.domain.id=default&role.id=6e0201a9a6484f7d8b112b1eafff70bd => generated 233 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:32:19.358238 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8985e766-bd2a-4c38-b84b-405ee7f3a1a5 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:19.358749 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8985e766-bd2a-4c38-b84b-405ee7f3a1a5 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:19.358847 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8985e766-bd2a-4c38-b84b-405ee7f3a1a5 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:19.360341 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 11/22] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:32:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:32:19.365763 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fad7a6-04a6-4361-8a57-47c0e26b635f None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:19.366216 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fad7a6-04a6-4361-8a57-47c0e26b635f None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:19.366418 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fad7a6-04a6-4361-8a57-47c0e26b635f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:19.394582 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-a8fad7a6-04a6-4361-8a57-47c0e26b635f None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:32:19.395819 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-a8fad7a6-04a6-4361-8a57-47c0e26b635f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:32:19.489461 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 12/23] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:32:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 13:32:19.539031 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8985e766-bd2a-4c38-b84b-405ee7f3a1a5 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:19.691153 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8985e766-bd2a-4c38-b84b-405ee7f3a1a5 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:19.700763 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ecbf7e-39c0-4484-8d4c-a357716903b8 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:19.701064 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ecbf7e-39c0-4484-8d4c-a357716903b8 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:19.701414 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ecbf7e-39c0-4484-8d4c-a357716903b8 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:19.703348 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 13/24] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:32:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:32:19.717710 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2042ecca-0cc6-4388-b7fd-b41ae740951b None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:19.717996 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2042ecca-0cc6-4388-b7fd-b41ae740951b None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:19.718284 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2042ecca-0cc6-4388-b7fd-b41ae740951b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:19.731795 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:19.755106 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-2042ecca-0cc6-4388-b7fd-b41ae740951b None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:32:19.756483 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-2042ecca-0cc6-4388-b7fd-b41ae740951b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:32:19.761998 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bc6910-5dbd-42ca-83c0-d9aaca42f340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:19.762664 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bc6910-5dbd-42ca-83c0-d9aaca42f340 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:19.762946 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bc6910-5dbd-42ca-83c0-d9aaca42f340 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:19.763254 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bc6910-5dbd-42ca-83c0-d9aaca42f340 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:19.787402 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bc6910-5dbd-42ca-83c0-d9aaca42f340 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:19.788895 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bc6910-5dbd-42ca-83c0-d9aaca42f340 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:19.816750 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-78bc6910-5dbd-42ca-83c0-d9aaca42f340 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:32:19.816750 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:32:19.816750 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:32:19.816750 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:32:19.816750 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:32:19.816750 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:32:19.816750 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:32:19.816750 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:32:19.816750 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:32:19.816750 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:32:19.816750 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:32:19.816750 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:32:19.816750 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 13:32:19.816750 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 13:32:19.816750 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 13:32:19.817643 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 13:32:19.817643 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:32:19.817643 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:32:19.817643 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:32:19.817643 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:32:19.817643 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:32:19.817643 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:32:19.817643 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:32:19.817643 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:32:19.817643 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:32:19.817643 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:32:19.817643 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:32:19.817643 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:32:19.817643 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:32:19.817643 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:32:19.817643 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:32:19.817643 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:32:19.818464 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 13:32:19.818464 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 13:32:19.818464 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:32:19.818464 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:32:19.818464 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 12/25] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:32:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:32:19.825540 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bc6910-5dbd-42ca-83c0-d9aaca42f340 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:19.895048 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 14/26] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:32:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 13:32:19.947766 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2042ecca-0cc6-4388-b7fd-b41ae740951b None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:19.950930 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:20.052378 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bc6910-5dbd-42ca-83c0-d9aaca42f340 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:20.103108 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0f81d1-8dc3-460e-bbb0-b78402f03b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:20.104007 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0f81d1-8dc3-460e-bbb0-b78402f03b89 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:20.104292 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2042ecca-0cc6-4388-b7fd-b41ae740951b None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:20.104427 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0f81d1-8dc3-460e-bbb0-b78402f03b89 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:20.104703 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0f81d1-8dc3-460e-bbb0-b78402f03b89 None admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:20.105402 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0f81d1-8dc3-460e-bbb0-b78402f03b89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:32:20.105811 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0f81d1-8dc3-460e-bbb0-b78402f03b89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:20.108854 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0f81d1-8dc3-460e-bbb0-b78402f03b89 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:20.124131 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 13/27] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Jul 26 13:32:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:20.125624 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402e75b0-37b5-4162-ae1f-4260db49cc34 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:20.126051 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402e75b0-37b5-4162-ae1f-4260db49cc34 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:20.126495 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402e75b0-37b5-4162-ae1f-4260db49cc34 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:20.127856 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 14/28] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:32:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:32:20.129453 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2197cfb-34a1-4d93-8475-5fe123ded6a8 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:20.129895 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2197cfb-34a1-4d93-8475-5fe123ded6a8 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:20.130303 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2197cfb-34a1-4d93-8475-5fe123ded6a8 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:20.131756 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 15/29] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:32:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:32:20.136969 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2197cfb-34a1-4d93-8475-5fe123ded6a8 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:20.152025 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a03b43-a342-4b15-bc67-68e8675d2ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:20.153158 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a03b43-a342-4b15-bc67-68e8675d2ef0 None admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:20.153649 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a03b43-a342-4b15-bc67-68e8675d2ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:20.154100 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a03b43-a342-4b15-bc67-68e8675d2ef0 None admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:20.155059 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a03b43-a342-4b15-bc67-68e8675d2ef0 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:20.156650 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a03b43-a342-4b15-bc67-68e8675d2ef0 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:20.281297 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:20.294804 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 15/30] 10.222.0.58 () {60 vars in 1131 bytes} [Tue Jul 26 13:32:19 2022] POST /identity/v3/roles => generated 219 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:32:20.297256 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c3654f-83d7-4e23-841a-101b0ba979a0 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:20.297774 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c3654f-83d7-4e23-841a-101b0ba979a0 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:20.298199 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c3654f-83d7-4e23-841a-101b0ba979a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:20.318632 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-54c3654f-83d7-4e23-841a-101b0ba979a0 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:32:20.320657 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-54c3654f-83d7-4e23-841a-101b0ba979a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:32:20.425087 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:20.438076 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 16/31] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:32:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 13:32:20.439392 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc82355d-672f-4c6d-9e31-9ba78d8ad129 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:20.439684 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc82355d-672f-4c6d-9e31-9ba78d8ad129 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:20.439970 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc82355d-672f-4c6d-9e31-9ba78d8ad129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:20.469360 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2197cfb-34a1-4d93-8475-5fe123ded6a8 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:20.481171 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-bc82355d-672f-4c6d-9e31-9ba78d8ad129 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:32:20.483235 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-bc82355d-672f-4c6d-9e31-9ba78d8ad129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:32:20.533231 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25d96bc-8060-4e02-9322-58a70aaa1350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:20.533835 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25d96bc-8060-4e02-9322-58a70aaa1350 None admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:20.534121 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25d96bc-8060-4e02-9322-58a70aaa1350 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:20.534397 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25d96bc-8060-4e02-9322-58a70aaa1350 None admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:20.535391 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25d96bc-8060-4e02-9322-58a70aaa1350 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:20.537195 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25d96bc-8060-4e02-9322-58a70aaa1350 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:20.585564 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 17/32] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:32:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 13:32:20.586535 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc82355d-672f-4c6d-9e31-9ba78d8ad129 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:20.719781 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc82355d-672f-4c6d-9e31-9ba78d8ad129 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:20.745932 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:20.757792 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbaf768-84bd-452c-a2af-890c6375b60b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:20.758623 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbaf768-84bd-452c-a2af-890c6375b60b None admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:20.759056 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbaf768-84bd-452c-a2af-890c6375b60b None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:20.759510 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbaf768-84bd-452c-a2af-890c6375b60b None admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:20.760375 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbaf768-84bd-452c-a2af-890c6375b60b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:20.762068 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbaf768-84bd-452c-a2af-890c6375b60b None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:20.778904 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 18/33] 10.222.0.58 () {60 vars in 1131 bytes} [Tue Jul 26 13:32:20 2022] POST /identity/v3/roles => generated 225 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:32:20.780094 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbaf768-84bd-452c-a2af-890c6375b60b None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:20.819866 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 16/34] 10.222.0.58 () {60 vars in 1141 bytes} [Tue Jul 26 13:32:20 2022] POST /identity/v3/projects => generated 301 bytes in 683 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:32:20.905258 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbaf768-84bd-452c-a2af-890c6375b60b None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:20.934702 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad6f572-e3c9-4de0-b77d-0b638c2122ac None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:20.934999 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad6f572-e3c9-4de0-b77d-0b638c2122ac None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:20.935290 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad6f572-e3c9-4de0-b77d-0b638c2122ac None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:20.937384 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 17/35] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:32:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:32:20.943286 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491fbd3c-4d93-4481-b549-016ba57ba4e0 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:20.943515 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491fbd3c-4d93-4481-b549-016ba57ba4e0 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:20.943848 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491fbd3c-4d93-4481-b549-016ba57ba4e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:20.958400 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b015878-e849-4c4e-970e-49046f6f734c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:20.959095 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b015878-e849-4c4e-970e-49046f6f734c None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:20.959389 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b015878-e849-4c4e-970e-49046f6f734c None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:20.959675 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b015878-e849-4c4e-970e-49046f6f734c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:20.968122 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-491fbd3c-4d93-4481-b549-016ba57ba4e0 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:32:20.974439 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-491fbd3c-4d93-4481-b549-016ba57ba4e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:32:20.987755 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b015878-e849-4c4e-970e-49046f6f734c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:32:20.996262 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-5b015878-e849-4c4e-970e-49046f6f734c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 13:32:20.996262 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:32:20.996262 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:32:20.996262 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:32:20.996262 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:32:20.996262 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:32:20.996262 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:32:20.996262 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:32:20.996262 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:32:20.996262 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:32:20.996262 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:32:20.996262 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:32:20.996262 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:32:20.996262 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:32:20.996262 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:32:20.996262 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:32:20.997358 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:32:20.997358 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:32:20.997358 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:32:20.997358 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:32:20.997358 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:32:20.997358 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:32:20.997358 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:32:20.997358 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:32:20.997358 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:32:20.997358 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:32:20.997358 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:32:20.997358 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:32:20.997358 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:32:20.997358 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:32:20.997358 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:32:20.997358 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:32:20.998357 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:32:20.998357 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:32:20.998357 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:32:20.998357 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:32:20.998357 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:32:20.998357 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:32:20.998357 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:32:20.998357 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:32:20.998357 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:32:20.998357 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:32:20.998357 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 13:32:20.998357 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:32:21.002864 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 19/36] 10.222.0.58 () {56 vars in 1096 bytes} [Tue Jul 26 13:32:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:32:21.009444 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b015878-e849-4c4e-970e-49046f6f734c None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:21.084867 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:21.096427 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 18/37] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:32:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 13:32:21.121764 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491fbd3c-4d93-4481-b549-016ba57ba4e0 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:21.140261 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:21.175549 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b015878-e849-4c4e-970e-49046f6f734c None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:21.214999 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6572b6-8dd8-4fef-bae8-c5a5186211d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:21.215630 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6572b6-8dd8-4fef-bae8-c5a5186211d9 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:21.215915 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6572b6-8dd8-4fef-bae8-c5a5186211d9 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:21.216185 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6572b6-8dd8-4fef-bae8-c5a5186211d9 None admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:21.217515 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6572b6-8dd8-4fef-bae8-c5a5186211d9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:32:21.217765 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6572b6-8dd8-4fef-bae8-c5a5186211d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:21.219154 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6572b6-8dd8-4fef-bae8-c5a5186211d9 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:21.228034 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 20/38] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Jul 26 13:32:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:21.230870 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f169beea-9eea-4a23-beac-298b3dcfab38 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:21.231359 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f169beea-9eea-4a23-beac-298b3dcfab38 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:21.231653 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f169beea-9eea-4a23-beac-298b3dcfab38 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:21.233514 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 21/39] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:32:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 13:32:21.234231 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f169beea-9eea-4a23-beac-298b3dcfab38 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:21.253054 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491fbd3c-4d93-4481-b549-016ba57ba4e0 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:21.288480 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cc40e4-68fe-4007-be07-27276b222d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:21.289108 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cc40e4-68fe-4007-be07-27276b222d6d None admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:21.289399 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cc40e4-68fe-4007-be07-27276b222d6d None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:21.289700 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cc40e4-68fe-4007-be07-27276b222d6d None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:21.290428 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cc40e4-68fe-4007-be07-27276b222d6d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:21.291942 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cc40e4-68fe-4007-be07-27276b222d6d None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:21.308971 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 19/40] 10.222.0.58 () {60 vars in 1131 bytes} [Tue Jul 26 13:32:21 2022] POST /identity/v3/roles => generated 223 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:32:21.310217 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6baa7b-4257-4d5d-9e91-98a5c5296749 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:21.310507 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6baa7b-4257-4d5d-9e91-98a5c5296749 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:21.310782 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6baa7b-4257-4d5d-9e91-98a5c5296749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:21.331675 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-1e6baa7b-4257-4d5d-9e91-98a5c5296749 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:32:21.332918 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-1e6baa7b-4257-4d5d-9e91-98a5c5296749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:32:21.345373 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f169beea-9eea-4a23-beac-298b3dcfab38 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:21.353820 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:21.388921 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee93008f-00ca-4917-a859-4735336330b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:21.390272 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee93008f-00ca-4917-a859-4735336330b9 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:21.390563 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee93008f-00ca-4917-a859-4735336330b9 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:21.390842 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee93008f-00ca-4917-a859-4735336330b9 None admin] PATH_INFO: `/v3/users/f1ff1f65366f4d20a56500bb6bc88c54` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:21.392594 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee93008f-00ca-4917-a859-4735336330b9 None admin] RBAC: Authorizing `identity:get_user(user_id=f1ff1f65366f4d20a56500bb6bc88c54)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:21.394531 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee93008f-00ca-4917-a859-4735336330b9 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:21.401267 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 22/41] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Jul 26 13:32:21 2022] GET /identity/v3/users/f1ff1f65366f4d20a56500bb6bc88c54 => generated 247 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:21.402276 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee93008f-00ca-4917-a859-4735336330b9 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:21.408215 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:21.417515 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 20/42] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:32:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 13:32:21.438152 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6baa7b-4257-4d5d-9e91-98a5c5296749 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:21.535001 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee93008f-00ca-4917-a859-4735336330b9 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:21.553752 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6baa7b-4257-4d5d-9e91-98a5c5296749 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:21.568874 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783a5178-9296-4453-a188-06cf39b167ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:21.569538 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783a5178-9296-4453-a188-06cf39b167ab None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:21.569824 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783a5178-9296-4453-a188-06cf39b167ab None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:21.570097 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783a5178-9296-4453-a188-06cf39b167ab None admin] PATH_INFO: `/v3/domains/default` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:21.571252 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783a5178-9296-4453-a188-06cf39b167ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:21.572690 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783a5178-9296-4453-a188-06cf39b167ab None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:21.574297 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 23/43] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:32:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:21.580483 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783a5178-9296-4453-a188-06cf39b167ab None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:21.589273 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b010d77b-5dca-4a2e-8ecf-2d67696503e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:21.589960 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b010d77b-5dca-4a2e-8ecf-2d67696503e2 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:21.590233 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b010d77b-5dca-4a2e-8ecf-2d67696503e2 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:21.590509 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b010d77b-5dca-4a2e-8ecf-2d67696503e2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:21.591672 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b010d77b-5dca-4a2e-8ecf-2d67696503e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:21.593748 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b010d77b-5dca-4a2e-8ecf-2d67696503e2 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:21.599824 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 21/44] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:32:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:32:21.601018 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b010d77b-5dca-4a2e-8ecf-2d67696503e2 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:21.632103 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:21.667268 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:21.691101 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783a5178-9296-4453-a188-06cf39b167ab None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:21.703063 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b010d77b-5dca-4a2e-8ecf-2d67696503e2 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:21.726134 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d588914-f353-4449-80d8-437baf851554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:21.726815 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d588914-f353-4449-80d8-437baf851554 None admin] REQUEST_METHOD: `PUT` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:21.727102 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d588914-f353-4449-80d8-437baf851554 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:21.727384 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d588914-f353-4449-80d8-437baf851554 None admin] PATH_INFO: `/v3/domains/default/users/f1ff1f65366f4d20a56500bb6bc88c54/roles/6e0201a9a6484f7d8b112b1eafff70bd` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:21.729390 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d588914-f353-4449-80d8-437baf851554 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=f1ff1f65366f4d20a56500bb6bc88c54, role_id=6e0201a9a6484f7d8b112b1eafff70bd)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:21.730876 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d588914-f353-4449-80d8-437baf851554 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:21.737721 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4627045a-6983-4a18-98ec-2760ebbe95a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:21.738446 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4627045a-6983-4a18-98ec-2760ebbe95a4 None admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:21.738771 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4627045a-6983-4a18-98ec-2760ebbe95a4 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:21.739043 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4627045a-6983-4a18-98ec-2760ebbe95a4 None admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:21.739854 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4627045a-6983-4a18-98ec-2760ebbe95a4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:21.741346 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4627045a-6983-4a18-98ec-2760ebbe95a4 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:21.745035 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 24/45] 10.222.0.58 () {58 vars in 1361 bytes} [Tue Jul 26 13:32:21 2022] PUT /identity/v3/domains/default/users/f1ff1f65366f4d20a56500bb6bc88c54/roles/6e0201a9a6484f7d8b112b1eafff70bd => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 13:32:21.771613 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 22/46] 10.222.0.58 () {60 vars in 1141 bytes} [Tue Jul 26 13:32:21 2022] POST /identity/v3/projects => generated 312 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 13:32:23.764862 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90edd79-5385-4e9e-b23c-4327eaa5e8b5 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:23.765393 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90edd79-5385-4e9e-b23c-4327eaa5e8b5 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:23.765498 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90edd79-5385-4e9e-b23c-4327eaa5e8b5 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:23.766825 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 25/47] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:32:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:32:23.771987 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0071d5b9-4e62-411d-9e9c-0b002277d201 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:23.772397 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0071d5b9-4e62-411d-9e9c-0b002277d201 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:23.772673 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0071d5b9-4e62-411d-9e9c-0b002277d201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:23.791401 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-0071d5b9-4e62-411d-9e9c-0b002277d201 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:32:23.792587 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-0071d5b9-4e62-411d-9e9c-0b002277d201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:32:23.823117 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4a63d8-c45b-4a4d-a676-53c9e48b96d1 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:23.823413 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4a63d8-c45b-4a4d-a676-53c9e48b96d1 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:23.823697 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4a63d8-c45b-4a4d-a676-53c9e48b96d1 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:23.825061 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 26/48] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:32:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 13:32:23.830310 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8817cb27-e899-478f-94ac-62e86a2fbaf8 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:23.830600 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8817cb27-e899-478f-94ac-62e86a2fbaf8 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:23.830881 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8817cb27-e899-478f-94ac-62e86a2fbaf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:23.849024 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-8817cb27-e899-478f-94ac-62e86a2fbaf8 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:32:23.850366 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-8817cb27-e899-478f-94ac-62e86a2fbaf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:32:23.860240 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 23/49] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:32:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 13:32:23.882966 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0071d5b9-4e62-411d-9e9c-0b002277d201 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:23.916733 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 27/50] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:32:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 13:32:23.933530 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:23.940218 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8817cb27-e899-478f-94ac-62e86a2fbaf8 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:23.972116 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:23.988572 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0071d5b9-4e62-411d-9e9c-0b002277d201 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:24.023706 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d5e14e-e2eb-4d62-88d0-2bd183494c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:24.024270 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d5e14e-e2eb-4d62-88d0-2bd183494c59 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:24.024552 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d5e14e-e2eb-4d62-88d0-2bd183494c59 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:24.024836 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d5e14e-e2eb-4d62-88d0-2bd183494c59 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:24.036255 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d5e14e-e2eb-4d62-88d0-2bd183494c59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:32:24.042830 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-b9d5e14e-e2eb-4d62-88d0-2bd183494c59 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 13:32:24.042830 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:32:24.042830 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:32:24.042830 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:32:24.042830 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:32:24.042830 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:32:24.042830 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:32:24.042830 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:32:24.042830 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:32:24.042830 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:32:24.042830 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:32:24.042830 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:32:24.042830 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:32:24.042830 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:32:24.042830 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:32:24.042830 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:32:24.043646 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:32:24.043646 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:32:24.043646 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:32:24.043646 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:32:24.043646 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:32:24.043646 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:32:24.043646 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:32:24.043646 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:32:24.043646 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:32:24.043646 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:32:24.043646 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:32:24.043646 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:32:24.043646 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:32:24.043646 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:32:24.043646 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:32:24.043646 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:32:24.044397 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:32:24.044397 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:32:24.044397 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:32:24.044397 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:32:24.044397 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:32:24.044397 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:32:24.044397 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:32:24.044397 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:32:24.044397 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:32:24.044397 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:32:24.044397 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 13:32:24.044397 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:32:24.044397 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 24/51] 10.222.0.58 () {56 vars in 1096 bytes} [Tue Jul 26 13:32:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 13:32:24.046259 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8817cb27-e899-478f-94ac-62e86a2fbaf8 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:24.049722 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d5e14e-e2eb-4d62-88d0-2bd183494c59 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:24.082110 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb6818f-5f79-43c1-b7db-66c1f98bafdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:24.082834 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb6818f-5f79-43c1-b7db-66c1f98bafdd None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:24.083134 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb6818f-5f79-43c1-b7db-66c1f98bafdd None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:24.083416 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb6818f-5f79-43c1-b7db-66c1f98bafdd None admin] PATH_INFO: `/v3/domains/default` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:24.084592 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb6818f-5f79-43c1-b7db-66c1f98bafdd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:24.086086 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb6818f-5f79-43c1-b7db-66c1f98bafdd None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:24.087862 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 28/52] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:32:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:32:24.093352 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb6818f-5f79-43c1-b7db-66c1f98bafdd None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:24.158668 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d5e14e-e2eb-4d62-88d0-2bd183494c59 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:24.169278 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:24.193339 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:24.193896 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d20e1e-b07f-4cb2-8c73-de5c791a173d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:24.194567 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d20e1e-b07f-4cb2-8c73-de5c791a173d None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:24.194921 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d20e1e-b07f-4cb2-8c73-de5c791a173d None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:24.195214 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d20e1e-b07f-4cb2-8c73-de5c791a173d None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:24.195983 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d20e1e-b07f-4cb2-8c73-de5c791a173d None admin] RBAC: Adding query filter params (name=admin) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:32:24.196328 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d20e1e-b07f-4cb2-8c73-de5c791a173d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:24.196558 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb6818f-5f79-43c1-b7db-66c1f98bafdd None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:24.197785 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d20e1e-b07f-4cb2-8c73-de5c791a173d None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:24.210637 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 25/53] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Jul 26 13:32:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:24.216173 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d20e1e-b07f-4cb2-8c73-de5c791a173d None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:24.232328 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638a2989-9cbd-458c-9f0d-4025a2425218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:24.233307 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638a2989-9cbd-458c-9f0d-4025a2425218 None admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:24.233625 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638a2989-9cbd-458c-9f0d-4025a2425218 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:24.233910 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638a2989-9cbd-458c-9f0d-4025a2425218 None admin] PATH_INFO: `/v3/projects` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:24.234590 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638a2989-9cbd-458c-9f0d-4025a2425218 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:24.236016 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638a2989-9cbd-458c-9f0d-4025a2425218 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:24.266807 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 29/54] 10.222.0.58 () {60 vars in 1140 bytes} [Tue Jul 26 13:32:24 2022] POST /identity/v3/projects => generated 298 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:32:24.332537 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d20e1e-b07f-4cb2-8c73-de5c791a173d None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:24.369988 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2a5e09-28dc-4d65-bde1-416ffbc5d764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:24.370874 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2a5e09-28dc-4d65-bde1-416ffbc5d764 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:24.371161 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2a5e09-28dc-4d65-bde1-416ffbc5d764 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:24.371440 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2a5e09-28dc-4d65-bde1-416ffbc5d764 None admin] PATH_INFO: `/v3/users/f1ff1f65366f4d20a56500bb6bc88c54` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:24.373574 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2a5e09-28dc-4d65-bde1-416ffbc5d764 None admin] RBAC: Authorizing `identity:get_user(user_id=f1ff1f65366f4d20a56500bb6bc88c54)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:24.375078 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2a5e09-28dc-4d65-bde1-416ffbc5d764 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:24.376753 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 26/55] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Jul 26 13:32:24 2022] GET /identity/v3/users/f1ff1f65366f4d20a56500bb6bc88c54 => generated 247 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:32:24.382531 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638a2989-9cbd-458c-9f0d-4025a2425218 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:24.487953 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638a2989-9cbd-458c-9f0d-4025a2425218 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:24.522700 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029fe0b1-8efe-4824-9559-e29a490eb032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:24.523357 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029fe0b1-8efe-4824-9559-e29a490eb032 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:24.523648 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029fe0b1-8efe-4824-9559-e29a490eb032 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:24.523932 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029fe0b1-8efe-4824-9559-e29a490eb032 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:24.525093 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029fe0b1-8efe-4824-9559-e29a490eb032 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:24.526572 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029fe0b1-8efe-4824-9559-e29a490eb032 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:24.528168 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 30/56] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:32:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:24.534338 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2a5e09-28dc-4d65-bde1-416ffbc5d764 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:24.577095 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:24.636149 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2a5e09-28dc-4d65-bde1-416ffbc5d764 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:24.669018 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bbb023-f276-4aaa-adc6-c77aa4694c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:24.669668 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bbb023-f276-4aaa-adc6-c77aa4694c5e None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:24.670520 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bbb023-f276-4aaa-adc6-c77aa4694c5e None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:24.670811 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bbb023-f276-4aaa-adc6-c77aa4694c5e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:24.671594 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bbb023-f276-4aaa-adc6-c77aa4694c5e None admin] RBAC: Adding query filter params (role.id=6e0201a9a6484f7d8b112b1eafff70bd, scope.domain.id=default, user.id=f1ff1f65366f4d20a56500bb6bc88c54) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:32:24.671909 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bbb023-f276-4aaa-adc6-c77aa4694c5e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:24.673369 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bbb023-f276-4aaa-adc6-c77aa4694c5e None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:24.683518 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 27/57] 10.222.0.58 () {56 vars in 1322 bytes} [Tue Jul 26 13:32:24 2022] GET /identity/v3/role_assignments?user.id=f1ff1f65366f4d20a56500bb6bc88c54&scope.domain.id=default&role.id=6e0201a9a6484f7d8b112b1eafff70bd => generated 530 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:26.299107 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b067fe-3ec6-4324-a9ab-7d5b29d0a730 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:26.299591 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b067fe-3ec6-4324-a9ab-7d5b29d0a730 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:26.299591 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b067fe-3ec6-4324-a9ab-7d5b29d0a730 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:26.300940 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 31/58] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:32:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:32:26.306263 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f9dc50-d6ed-49e3-bf73-0566d9a0789c None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:26.306653 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f9dc50-d6ed-49e3-bf73-0566d9a0789c None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:26.306824 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f9dc50-d6ed-49e3-bf73-0566d9a0789c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:26.324493 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-c4f9dc50-d6ed-49e3-bf73-0566d9a0789c None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:32:26.325717 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-c4f9dc50-d6ed-49e3-bf73-0566d9a0789c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:32:26.389667 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 28/59] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:32:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 13:32:26.413397 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b067fe-3ec6-4324-a9ab-7d5b29d0a730 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:26.435757 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:26.513860 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b067fe-3ec6-4324-a9ab-7d5b29d0a730 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:26.546527 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4de5888-e11d-450c-aa5a-d47e40a206e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:26.547165 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4de5888-e11d-450c-aa5a-d47e40a206e9 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:26.547455 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4de5888-e11d-450c-aa5a-d47e40a206e9 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:26.547738 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4de5888-e11d-450c-aa5a-d47e40a206e9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:26.548897 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4de5888-e11d-450c-aa5a-d47e40a206e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:26.550433 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4de5888-e11d-450c-aa5a-d47e40a206e9 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:26.552238 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 32/60] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:32:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:26.557707 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f9dc50-d6ed-49e3-bf73-0566d9a0789c None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:26.590571 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:26.656111 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f9dc50-d6ed-49e3-bf73-0566d9a0789c None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:26.688751 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b3fa02-dabd-4075-962d-95bd6235f05e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:26.689368 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b3fa02-dabd-4075-962d-95bd6235f05e None admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:26.689684 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b3fa02-dabd-4075-962d-95bd6235f05e None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:26.689968 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b3fa02-dabd-4075-962d-95bd6235f05e None admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:26.690664 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b3fa02-dabd-4075-962d-95bd6235f05e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:26.692231 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b3fa02-dabd-4075-962d-95bd6235f05e None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:26.719751 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 29/61] 10.222.0.58 () {60 vars in 1132 bytes} [Tue Jul 26 13:32:26 2022] POST /identity/v3/users => generated 275 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 13:32:29.911381 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca24eea-1431-4549-a24d-79adf230ffbd None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:29.912412 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca24eea-1431-4549-a24d-79adf230ffbd None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:29.912934 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca24eea-1431-4549-a24d-79adf230ffbd None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:29.914570 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 33/62] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:32:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:32:29.921252 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d16d84-3927-476b-b1b5-759abc5d3869 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:29.922183 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d16d84-3927-476b-b1b5-759abc5d3869 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:29.922665 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d16d84-3927-476b-b1b5-759abc5d3869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:29.950615 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-83d16d84-3927-476b-b1b5-759abc5d3869 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:32:29.952232 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-83d16d84-3927-476b-b1b5-759abc5d3869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:32:30.043077 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdf4923-00b2-464c-b516-1f024093cd80 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:30.043648 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdf4923-00b2-464c-b516-1f024093cd80 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:30.044155 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdf4923-00b2-464c-b516-1f024093cd80 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:30.046440 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 34/63] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:32:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:32:30.056052 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 30/64] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:32:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 13:32:30.058894 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ad5d52-5a5b-433b-a9a3-39ac73d486e4 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:30.059409 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ad5d52-5a5b-433b-a9a3-39ac73d486e4 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:30.059891 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ad5d52-5a5b-433b-a9a3-39ac73d486e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:30.079234 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d16d84-3927-476b-b1b5-759abc5d3869 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:30.090416 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-47ad5d52-5a5b-433b-a9a3-39ac73d486e4 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:32:30.092186 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-47ad5d52-5a5b-433b-a9a3-39ac73d486e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:32:30.102521 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:30.188980 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 35/65] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:32:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 13:32:30.190522 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706a8ecb-bb52-4930-8373-6ab4b4c3dba9 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:30.190974 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706a8ecb-bb52-4930-8373-6ab4b4c3dba9 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:30.191410 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706a8ecb-bb52-4930-8373-6ab4b4c3dba9 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:30.196956 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 36/66] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:32:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:32:30.198730 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc0c110-14cf-45a0-a715-deb66f0fdddf None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:30.199402 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc0c110-14cf-45a0-a715-deb66f0fdddf None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:30.199822 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc0c110-14cf-45a0-a715-deb66f0fdddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:30.221289 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-6fc0c110-14cf-45a0-a715-deb66f0fdddf None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:32:30.223290 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-6fc0c110-14cf-45a0-a715-deb66f0fdddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:32:30.231468 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d16d84-3927-476b-b1b5-759abc5d3869 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:30.242609 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:30.275410 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a60970-47ef-454e-9329-1e9ec66aa8e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:30.276377 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a60970-47ef-454e-9329-1e9ec66aa8e3 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:30.276908 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a60970-47ef-454e-9329-1e9ec66aa8e3 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:30.277328 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a60970-47ef-454e-9329-1e9ec66aa8e3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:30.289952 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a60970-47ef-454e-9329-1e9ec66aa8e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:32:30.295926 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-38a60970-47ef-454e-9329-1e9ec66aa8e3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 13:32:30.295926 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:32:30.295926 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:32:30.295926 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:32:30.295926 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:32:30.295926 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:32:30.295926 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:32:30.295926 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:32:30.295926 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:32:30.295926 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:32:30.295926 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:32:30.295926 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:32:30.295926 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:32:30.295926 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:32:30.295926 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:32:30.295926 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:32:30.296848 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:32:30.296848 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:32:30.296848 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:32:30.296848 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:32:30.296848 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:32:30.296848 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:32:30.296848 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:32:30.296848 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:32:30.296848 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:32:30.296848 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:32:30.296848 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:32:30.296848 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:32:30.296848 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:32:30.296848 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:32:30.296848 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:32:30.296848 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:32:30.297723 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:32:30.297723 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:32:30.297723 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:32:30.297723 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:32:30.297723 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:32:30.297723 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:32:30.297723 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:32:30.297723 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:32:30.297723 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:32:30.297723 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:32:30.297723 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 13:32:30.297723 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:32:30.301368 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 31/67] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Jul 26 13:32:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 13:32:30.302480 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a60970-47ef-454e-9329-1e9ec66aa8e3 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:30.323513 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 37/68] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:32:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 13:32:30.325118 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6a9598-ca64-474b-89e8-002c00eb9af6 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:30.326174 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6a9598-ca64-474b-89e8-002c00eb9af6 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:30.326820 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6a9598-ca64-474b-89e8-002c00eb9af6 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:30.328198 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 38/69] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:32:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:32:30.328918 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6a9598-ca64-474b-89e8-002c00eb9af6 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:30.401403 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:30.424273 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a60970-47ef-454e-9329-1e9ec66aa8e3 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:30.457254 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6a9598-ca64-474b-89e8-002c00eb9af6 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:30.462101 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc04b20-8d5e-4cf0-8d77-e22d3da78052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:30.462758 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc04b20-8d5e-4cf0-8d77-e22d3da78052 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:30.463051 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc04b20-8d5e-4cf0-8d77-e22d3da78052 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:30.463332 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc04b20-8d5e-4cf0-8d77-e22d3da78052 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:30.475701 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc04b20-8d5e-4cf0-8d77-e22d3da78052 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:32:30.480943 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-fbc04b20-8d5e-4cf0-8d77-e22d3da78052 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 13:32:30.480943 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:32:30.480943 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:32:30.480943 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:32:30.480943 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:32:30.480943 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:32:30.480943 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:32:30.480943 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:32:30.480943 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:32:30.480943 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:32:30.480943 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:32:30.480943 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:32:30.480943 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:32:30.480943 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:32:30.480943 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:32:30.480943 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:32:30.482018 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:32:30.482018 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:32:30.482018 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:32:30.482018 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:32:30.482018 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:32:30.482018 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:32:30.482018 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:32:30.482018 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:32:30.482018 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:32:30.482018 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:32:30.482018 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:32:30.482018 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:32:30.482018 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:32:30.482018 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:32:30.482018 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:32:30.482018 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:32:30.482940 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:32:30.482940 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:32:30.482940 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:32:30.482940 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:32:30.482940 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:32:30.482940 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:32:30.482940 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:32:30.482940 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:32:30.482940 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:32:30.482940 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:32:30.482940 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 13:32:30.482940 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:32:30.484904 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 32/70] 10.222.0.58 () {56 vars in 1114 bytes} [Tue Jul 26 13:32:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:32:30.487511 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b5a3b2-a046-49f6-979b-f11567aae1da None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:30.488052 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b5a3b2-a046-49f6-979b-f11567aae1da None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:30.488709 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b5a3b2-a046-49f6-979b-f11567aae1da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:30.497616 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7c3cac-40f2-4be6-bd76-aefe5bbb1857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:30.498252 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7c3cac-40f2-4be6-bd76-aefe5bbb1857 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:30.498538 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7c3cac-40f2-4be6-bd76-aefe5bbb1857 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:30.498819 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7c3cac-40f2-4be6-bd76-aefe5bbb1857 None admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:30.499554 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7c3cac-40f2-4be6-bd76-aefe5bbb1857 None admin] RBAC: Adding query filter params (name=member) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:32:30.499865 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7c3cac-40f2-4be6-bd76-aefe5bbb1857 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:30.501269 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7c3cac-40f2-4be6-bd76-aefe5bbb1857 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:30.508018 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-84b5a3b2-a046-49f6-979b-f11567aae1da None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:32:30.509236 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-84b5a3b2-a046-49f6-979b-f11567aae1da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:32:30.510605 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 39/71] 10.222.0.58 () {56 vars in 1101 bytes} [Tue Jul 26 13:32:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:30.511287 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7c3cac-40f2-4be6-bd76-aefe5bbb1857 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:30.527207 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:30.590128 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 33/72] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:32:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 13:32:30.594442 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b5a3b2-a046-49f6-979b-f11567aae1da None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:30.622615 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7c3cac-40f2-4be6-bd76-aefe5bbb1857 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:30.656995 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db44f708-7c85-4c28-b332-8996e91e635e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:30.657678 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db44f708-7c85-4c28-b332-8996e91e635e None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:30.657960 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db44f708-7c85-4c28-b332-8996e91e635e None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:30.658236 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db44f708-7c85-4c28-b332-8996e91e635e None admin] PATH_INFO: `/v3/domains/default` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:30.662645 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db44f708-7c85-4c28-b332-8996e91e635e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:30.662645 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db44f708-7c85-4c28-b332-8996e91e635e None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:30.662645 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 40/73] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:32:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:30.663189 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db44f708-7c85-4c28-b332-8996e91e635e None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:30.671263 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:30.703496 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b5a3b2-a046-49f6-979b-f11567aae1da None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:30.740036 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b3b29d-8694-4772-a4ed-a59363c93a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:30.740586 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b3b29d-8694-4772-a4ed-a59363c93a0e None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:30.740877 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b3b29d-8694-4772-a4ed-a59363c93a0e None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:30.741159 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b3b29d-8694-4772-a4ed-a59363c93a0e None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:30.742032 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b3b29d-8694-4772-a4ed-a59363c93a0e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:32:30.742393 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b3b29d-8694-4772-a4ed-a59363c93a0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:30.744058 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b3b29d-8694-4772-a4ed-a59363c93a0e None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:30.753016 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:30.758783 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 34/74] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Jul 26 13:32:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:30.760073 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d751ff-c369-4787-8d91-f579aff8b6c6 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:30.760367 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d751ff-c369-4787-8d91-f579aff8b6c6 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:30.760650 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d751ff-c369-4787-8d91-f579aff8b6c6 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:30.762011 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 35/75] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:32:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:32:30.764911 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d751ff-c369-4787-8d91-f579aff8b6c6 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:30.783363 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db44f708-7c85-4c28-b332-8996e91e635e None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:30.823358 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bc05c6-8c16-4ac0-b0c7-a2b1e17e7932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:30.824018 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bc05c6-8c16-4ac0-b0c7-a2b1e17e7932 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:30.824407 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bc05c6-8c16-4ac0-b0c7-a2b1e17e7932 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:30.824700 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bc05c6-8c16-4ac0-b0c7-a2b1e17e7932 None admin] PATH_INFO: `/v3/users/eac6099386504fcea65a5d57e8a6c1d2` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:30.839150 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bc05c6-8c16-4ac0-b0c7-a2b1e17e7932 None admin] RBAC: Authorizing `identity:get_user(user_id=eac6099386504fcea65a5d57e8a6c1d2)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:30.840867 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bc05c6-8c16-4ac0-b0c7-a2b1e17e7932 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:30.842701 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 41/76] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Jul 26 13:32:30 2022] GET /identity/v3/users/eac6099386504fcea65a5d57e8a6c1d2 => generated 275 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:30.843516 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bc05c6-8c16-4ac0-b0c7-a2b1e17e7932 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:30.877290 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d751ff-c369-4787-8d91-f579aff8b6c6 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:30.907176 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:30.912811 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b61b9d-0d9d-480c-a78a-6d792dc446c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:30.913463 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b61b9d-0d9d-480c-a78a-6d792dc446c2 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:30.913855 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b61b9d-0d9d-480c-a78a-6d792dc446c2 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:30.914059 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b61b9d-0d9d-480c-a78a-6d792dc446c2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:30.925196 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b61b9d-0d9d-480c-a78a-6d792dc446c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:32:30.930427 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-01b61b9d-0d9d-480c-a78a-6d792dc446c2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 13:32:30.930427 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:32:30.930427 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:32:30.930427 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:32:30.930427 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:32:30.930427 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:32:30.930427 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:32:30.930427 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:32:30.930427 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:32:30.930427 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:32:30.930427 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:32:30.930427 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:32:30.930427 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:32:30.930427 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:32:30.930427 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:32:30.930427 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:32:30.931250 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:32:30.931250 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:32:30.931250 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:32:30.931250 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:32:30.931250 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:32:30.931250 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:32:30.931250 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:32:30.931250 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:32:30.931250 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:32:30.931250 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:32:30.931250 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:32:30.931250 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:32:30.931250 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:32:30.931250 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:32:30.931250 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:32:30.931250 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:32:30.932005 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:32:30.932005 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:32:30.932005 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:32:30.932005 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:32:30.932005 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:32:30.932005 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:32:30.932005 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:32:30.932005 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:32:30.932005 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:32:30.932005 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:32:30.932005 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 13:32:30.932005 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:32:30.932005 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 36/77] 10.222.0.58 () {56 vars in 1096 bytes} [Tue Jul 26 13:32:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:32:30.932630 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b61b9d-0d9d-480c-a78a-6d792dc446c2 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:30.959124 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bc05c6-8c16-4ac0-b0c7-a2b1e17e7932 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:31.006497 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668728fc-d611-4bbc-bfae-b8e930a91732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:31.007039 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668728fc-d611-4bbc-bfae-b8e930a91732 None admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:31.007319 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668728fc-d611-4bbc-bfae-b8e930a91732 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:31.007593 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668728fc-d611-4bbc-bfae-b8e930a91732 None admin] PATH_INFO: `/v3/users` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:31.008275 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668728fc-d611-4bbc-bfae-b8e930a91732 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:31.009727 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668728fc-d611-4bbc-bfae-b8e930a91732 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:31.022434 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:31.042194 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 42/78] 10.222.0.58 () {60 vars in 1132 bytes} [Tue Jul 26 13:32:30 2022] POST /identity/v3/users => generated 289 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:32:31.043558 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4430a5b9-a0e1-40cc-8bfe-a60e751c7fcc None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:31.043869 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4430a5b9-a0e1-40cc-8bfe-a60e751c7fcc None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:31.044180 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4430a5b9-a0e1-40cc-8bfe-a60e751c7fcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:31.067727 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-4430a5b9-a0e1-40cc-8bfe-a60e751c7fcc None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:32:31.068976 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-4430a5b9-a0e1-40cc-8bfe-a60e751c7fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:32:31.074810 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b61b9d-0d9d-480c-a78a-6d792dc446c2 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:31.124482 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8dab4a-218a-4c25-9979-5a37a3174af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:31.125447 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8dab4a-218a-4c25-9979-5a37a3174af3 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:31.125928 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8dab4a-218a-4c25-9979-5a37a3174af3 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:31.126364 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8dab4a-218a-4c25-9979-5a37a3174af3 None admin] PATH_INFO: `/v3/users/eac6099386504fcea65a5d57e8a6c1d2` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:31.128562 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8dab4a-218a-4c25-9979-5a37a3174af3 None admin] RBAC: Authorizing `identity:get_user(user_id=eac6099386504fcea65a5d57e8a6c1d2)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:31.131185 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8dab4a-218a-4c25-9979-5a37a3174af3 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:31.134192 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 37/79] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Jul 26 13:32:30 2022] GET /identity/v3/users/eac6099386504fcea65a5d57e8a6c1d2 => generated 275 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:31.135570 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8dab4a-218a-4c25-9979-5a37a3174af3 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:31.162778 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 43/80] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:32:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 13:32:31.163747 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4430a5b9-a0e1-40cc-8bfe-a60e751c7fcc None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:31.214624 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:31.270445 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8dab4a-218a-4c25-9979-5a37a3174af3 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:31.283756 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4430a5b9-a0e1-40cc-8bfe-a60e751c7fcc None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:31.315618 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:31.318957 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1315dd-511f-4b28-b721-60c709b5dabf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:31.319601 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1315dd-511f-4b28-b721-60c709b5dabf None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:31.319955 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1315dd-511f-4b28-b721-60c709b5dabf None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:31.320167 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1315dd-511f-4b28-b721-60c709b5dabf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:31.337187 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cc83f1-21f4-46d8-aec9-bc1d2aff1610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:31.338307 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cc83f1-21f4-46d8-aec9-bc1d2aff1610 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:31.339086 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cc83f1-21f4-46d8-aec9-bc1d2aff1610 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:31.339615 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cc83f1-21f4-46d8-aec9-bc1d2aff1610 None admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:31.341087 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cc83f1-21f4-46d8-aec9-bc1d2aff1610 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:32:31.341932 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cc83f1-21f4-46d8-aec9-bc1d2aff1610 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:31.344227 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cc83f1-21f4-46d8-aec9-bc1d2aff1610 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:31.356606 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 44/81] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Jul 26 13:32:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:31.359059 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cc83f1-21f4-46d8-aec9-bc1d2aff1610 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:31.409545 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1315dd-511f-4b28-b721-60c709b5dabf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:31.411150 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1315dd-511f-4b28-b721-60c709b5dabf None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:31.480827 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cc83f1-21f4-46d8-aec9-bc1d2aff1610 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:31.491097 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:31.529500 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c15016-474c-4464-afef-d641f065d24d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:31.530556 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c15016-474c-4464-afef-d641f065d24d None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:31.530846 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c15016-474c-4464-afef-d641f065d24d None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:31.531126 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c15016-474c-4464-afef-d641f065d24d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:31.542971 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 38/82] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Jul 26 13:32:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 13:32:31.543923 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1315dd-511f-4b28-b721-60c709b5dabf None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:31.595756 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:31.614777 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c15016-474c-4464-afef-d641f065d24d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:31.616375 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c15016-474c-4464-afef-d641f065d24d None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:31.669326 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1315dd-511f-4b28-b721-60c709b5dabf None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:31.708178 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c987f0-0ce3-488c-b08d-bc19a706f1da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:31.708902 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c987f0-0ce3-488c-b08d-bc19a706f1da None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:31.709312 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c987f0-0ce3-488c-b08d-bc19a706f1da None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:31.709672 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c987f0-0ce3-488c-b08d-bc19a706f1da None admin] PATH_INFO: `/v3/roles/member` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:31.723620 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c987f0-0ce3-488c-b08d-bc19a706f1da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:32:31.726392 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 45/83] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Jul 26 13:32:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 13:32:31.727524 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c15016-474c-4464-afef-d641f065d24d None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:31.729363 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-14c987f0-0ce3-488c-b08d-bc19a706f1da None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 13:32:31.729363 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:32:31.729363 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:32:31.729363 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:32:31.729363 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:32:31.729363 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:32:31.729363 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:32:31.729363 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:32:31.729363 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:32:31.729363 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:32:31.729363 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:32:31.729363 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:32:31.729363 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:32:31.729363 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:32:31.729363 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:32:31.729363 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:32:31.730440 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:32:31.730440 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:32:31.730440 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:32:31.730440 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:32:31.730440 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:32:31.730440 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:32:31.730440 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:32:31.730440 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:32:31.730440 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:32:31.730440 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:32:31.730440 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:32:31.730440 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:32:31.730440 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:32:31.730440 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:32:31.730440 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:32:31.730440 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:32:31.731382 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:32:31.731382 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:32:31.731382 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:32:31.731382 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:32:31.731382 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:32:31.731382 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:32:31.731382 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:32:31.731382 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:32:31.731382 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:32:31.731382 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:32:31.731382 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 13:32:31.731382 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:32:31.733509 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 39/84] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Jul 26 13:32:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:32:31.733861 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c987f0-0ce3-488c-b08d-bc19a706f1da None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:31.761438 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:31.848490 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:31.869179 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c15016-474c-4464-afef-d641f065d24d None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:31.876748 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c987f0-0ce3-488c-b08d-bc19a706f1da None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:31.908434 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2675806c-aeed-45f9-a3f7-03e78df7e165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:31.909094 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2675806c-aeed-45f9-a3f7-03e78df7e165 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:31.909376 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2675806c-aeed-45f9-a3f7-03e78df7e165 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:31.909716 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2675806c-aeed-45f9-a3f7-03e78df7e165 None admin] PATH_INFO: `/v3/users/f1ff1f65366f4d20a56500bb6bc88c54` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:31.910958 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2675806c-aeed-45f9-a3f7-03e78df7e165 None admin] RBAC: Authorizing `identity:get_user(user_id=f1ff1f65366f4d20a56500bb6bc88c54)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:31.912715 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2675806c-aeed-45f9-a3f7-03e78df7e165 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:31.914256 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 46/85] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Jul 26 13:32:31 2022] GET /identity/v3/users/f1ff1f65366f4d20a56500bb6bc88c54 => generated 247 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:31.915471 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2675806c-aeed-45f9-a3f7-03e78df7e165 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:31.940873 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffc0aca-546e-4aa5-aa67-6e08dd522b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:31.943417 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffc0aca-546e-4aa5-aa67-6e08dd522b93 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:31.944193 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffc0aca-546e-4aa5-aa67-6e08dd522b93 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:31.944951 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffc0aca-546e-4aa5-aa67-6e08dd522b93 None admin] PATH_INFO: `/v3/projects/afb9ca9466054e008e3b052866da363d` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:31.948049 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffc0aca-546e-4aa5-aa67-6e08dd522b93 None admin] RBAC: Authorizing `identity:get_project(project_id=afb9ca9466054e008e3b052866da363d)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:31.950525 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffc0aca-546e-4aa5-aa67-6e08dd522b93 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:31.953991 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 40/86] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Jul 26 13:32:31 2022] GET /identity/v3/projects/afb9ca9466054e008e3b052866da363d => generated 298 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:31.955643 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffc0aca-546e-4aa5-aa67-6e08dd522b93 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:32.033681 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2675806c-aeed-45f9-a3f7-03e78df7e165 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:32.075237 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3736bae-da2c-4e12-ba92-163f1d340455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:32.076496 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3736bae-da2c-4e12-ba92-163f1d340455 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:32.076889 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3736bae-da2c-4e12-ba92-163f1d340455 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:32.077514 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3736bae-da2c-4e12-ba92-163f1d340455 None admin] PATH_INFO: `/v3/projects/afb9ca9466054e008e3b052866da363d` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:32.078564 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3736bae-da2c-4e12-ba92-163f1d340455 None admin] RBAC: Authorizing `identity:get_project(project_id=afb9ca9466054e008e3b052866da363d)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:32.080138 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3736bae-da2c-4e12-ba92-163f1d340455 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:32.082151 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 47/87] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Jul 26 13:32:31 2022] GET /identity/v3/projects/afb9ca9466054e008e3b052866da363d => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:32.082795 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3736bae-da2c-4e12-ba92-163f1d340455 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:32.095523 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:32.110059 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:32.114564 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffc0aca-546e-4aa5-aa67-6e08dd522b93 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:32.151226 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4b4791-32f1-4793-8619-ef0e0f91d055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:32.151684 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4b4791-32f1-4793-8619-ef0e0f91d055 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:32.152014 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4b4791-32f1-4793-8619-ef0e0f91d055 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:32.152291 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4b4791-32f1-4793-8619-ef0e0f91d055 None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:32.153069 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4b4791-32f1-4793-8619-ef0e0f91d055 None admin] RBAC: Adding query filter params (name=member) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:32:32.153403 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4b4791-32f1-4793-8619-ef0e0f91d055 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:32.155445 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4b4791-32f1-4793-8619-ef0e0f91d055 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:32.164441 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 41/88] 10.222.0.58 () {56 vars in 1101 bytes} [Tue Jul 26 13:32:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:32.165129 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4b4791-32f1-4793-8619-ef0e0f91d055 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:32.228107 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3736bae-da2c-4e12-ba92-163f1d340455 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:32.274507 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87796ace-d978-42c0-995e-b631907a1001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:32.275382 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87796ace-d978-42c0-995e-b631907a1001 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:32.275742 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4b4791-32f1-4793-8619-ef0e0f91d055 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:32.275956 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87796ace-d978-42c0-995e-b631907a1001 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:32.276385 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87796ace-d978-42c0-995e-b631907a1001 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:32.324987 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c8f4b1-6244-4570-abd1-2bf07d8c6e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:32.327430 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c8f4b1-6244-4570-abd1-2bf07d8c6e10 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:32.327871 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c8f4b1-6244-4570-abd1-2bf07d8c6e10 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:32.328323 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c8f4b1-6244-4570-abd1-2bf07d8c6e10 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:32.329315 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c8f4b1-6244-4570-abd1-2bf07d8c6e10 None admin] RBAC: Adding query filter params (role.id=759f389b221049c4b01faf3b984cd8d7, scope.project.id=afb9ca9466054e008e3b052866da363d, user.id=eac6099386504fcea65a5d57e8a6c1d2) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:32:32.329807 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c8f4b1-6244-4570-abd1-2bf07d8c6e10 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:32.331463 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c8f4b1-6244-4570-abd1-2bf07d8c6e10 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:32.339896 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 42/89] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Jul 26 13:32:32 2022] GET /identity/v3/role_assignments?user.id=eac6099386504fcea65a5d57e8a6c1d2&scope.project.id=afb9ca9466054e008e3b052866da363d&role.id=759f389b221049c4b01faf3b984cd8d7 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:32.341224 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c8f4b1-6244-4570-abd1-2bf07d8c6e10 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:32.365707 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:32.373870 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87796ace-d978-42c0-995e-b631907a1001 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:32.375550 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87796ace-d978-42c0-995e-b631907a1001 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:32.423569 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:32.481983 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c8f4b1-6244-4570-abd1-2bf07d8c6e10 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:32.524562 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6edd60-a5d7-4fa1-8708-fe333a3d41f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:32.525497 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6edd60-a5d7-4fa1-8708-fe333a3d41f6 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:32.525925 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6edd60-a5d7-4fa1-8708-fe333a3d41f6 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:32.526342 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6edd60-a5d7-4fa1-8708-fe333a3d41f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:32.527301 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6edd60-a5d7-4fa1-8708-fe333a3d41f6 None admin] RBAC: Adding query filter params (role.id=5cb730ec786743c3bb8af83da90f9309, scope.project.id=afb9ca9466054e008e3b052866da363d, user.id=eac6099386504fcea65a5d57e8a6c1d2) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:32:32.527763 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6edd60-a5d7-4fa1-8708-fe333a3d41f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:32.529494 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6edd60-a5d7-4fa1-8708-fe333a3d41f6 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:32.536742 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 43/90] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Jul 26 13:32:32 2022] GET /identity/v3/role_assignments?user.id=eac6099386504fcea65a5d57e8a6c1d2&scope.project.id=afb9ca9466054e008e3b052866da363d&role.id=5cb730ec786743c3bb8af83da90f9309 => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:32.538015 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6edd60-a5d7-4fa1-8708-fe333a3d41f6 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:32.646483 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 48/91] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Jul 26 13:32:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 564 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 13:32:32.651057 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:32.655882 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87796ace-d978-42c0-995e-b631907a1001 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:32.692715 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6edd60-a5d7-4fa1-8708-fe333a3d41f6 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:32.761201 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e70673d-7f04-482a-bfc0-daf4c31fd545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:32.761918 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e70673d-7f04-482a-bfc0-daf4c31fd545 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:32.762305 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e70673d-7f04-482a-bfc0-daf4c31fd545 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:32.762473 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e70673d-7f04-482a-bfc0-daf4c31fd545 None admin] PATH_INFO: `/v3/users/eac6099386504fcea65a5d57e8a6c1d2` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:32.764027 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e70673d-7f04-482a-bfc0-daf4c31fd545 None admin] RBAC: Authorizing `identity:get_user(user_id=eac6099386504fcea65a5d57e8a6c1d2)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:32.765352 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e70673d-7f04-482a-bfc0-daf4c31fd545 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:32.767044 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 44/92] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Jul 26 13:32:32 2022] GET /identity/v3/users/eac6099386504fcea65a5d57e8a6c1d2 => generated 275 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:32.773274 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e70673d-7f04-482a-bfc0-daf4c31fd545 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:32.793204 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87796ace-d978-42c0-995e-b631907a1001 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:32.823397 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:32.873329 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecd5a0c-5828-44d0-aaa6-189af8cb94e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:32.874079 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecd5a0c-5828-44d0-aaa6-189af8cb94e8 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:32.874364 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecd5a0c-5828-44d0-aaa6-189af8cb94e8 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:32.874767 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecd5a0c-5828-44d0-aaa6-189af8cb94e8 None admin] PATH_INFO: `/v3/projects/afb9ca9466054e008e3b052866da363d` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:32.877774 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecd5a0c-5828-44d0-aaa6-189af8cb94e8 None admin] RBAC: Authorizing `identity:get_project(project_id=afb9ca9466054e008e3b052866da363d)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:32.879253 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecd5a0c-5828-44d0-aaa6-189af8cb94e8 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:32.881355 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 49/93] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Jul 26 13:32:32 2022] GET /identity/v3/projects/afb9ca9466054e008e3b052866da363d => generated 298 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:32.888076 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecd5a0c-5828-44d0-aaa6-189af8cb94e8 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:32.964697 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e70673d-7f04-482a-bfc0-daf4c31fd545 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:33.014975 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecd5a0c-5828-44d0-aaa6-189af8cb94e8 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:33.067275 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b444ba37-eeaf-4e48-afa5-6f8a7697d73c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:33.068011 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b444ba37-eeaf-4e48-afa5-6f8a7697d73c None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:33.068305 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b444ba37-eeaf-4e48-afa5-6f8a7697d73c None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:33.068590 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b444ba37-eeaf-4e48-afa5-6f8a7697d73c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:33.093453 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d245d7b2-0706-4ac3-aca3-0c7f551bda40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:33.094614 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d245d7b2-0706-4ac3-aca3-0c7f551bda40 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:33.094908 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d245d7b2-0706-4ac3-aca3-0c7f551bda40 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:33.095190 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d245d7b2-0706-4ac3-aca3-0c7f551bda40 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:33.096057 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d245d7b2-0706-4ac3-aca3-0c7f551bda40 None admin] RBAC: Adding query filter params (role.id=6e0201a9a6484f7d8b112b1eafff70bd, scope.project.id=afb9ca9466054e008e3b052866da363d, user.id=f1ff1f65366f4d20a56500bb6bc88c54) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:32:33.096376 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d245d7b2-0706-4ac3-aca3-0c7f551bda40 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:33.102831 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d245d7b2-0706-4ac3-aca3-0c7f551bda40 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:33.113797 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 50/94] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Jul 26 13:32:32 2022] GET /identity/v3/role_assignments?user.id=f1ff1f65366f4d20a56500bb6bc88c54&scope.project.id=afb9ca9466054e008e3b052866da363d&role.id=6e0201a9a6484f7d8b112b1eafff70bd => generated 259 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:32:33.115352 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:33.220006 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b444ba37-eeaf-4e48-afa5-6f8a7697d73c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:33.221466 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b444ba37-eeaf-4e48-afa5-6f8a7697d73c None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:33.377572 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 45/95] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Jul 26 13:32:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 605 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 13:32:33.386304 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d245d7b2-0706-4ac3-aca3-0c7f551bda40 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:33.483493 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:33.560527 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d245d7b2-0706-4ac3-aca3-0c7f551bda40 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:33.607112 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ab82b0-bf5a-4d05-88d4-66d139633c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:33.607792 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ab82b0-bf5a-4d05-88d4-66d139633c9d None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:33.608084 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ab82b0-bf5a-4d05-88d4-66d139633c9d None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:33.608368 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ab82b0-bf5a-4d05-88d4-66d139633c9d None admin] PATH_INFO: `/v3/projects/402eb261fe224c1faa856398d15ee4e2` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:33.609624 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ab82b0-bf5a-4d05-88d4-66d139633c9d None admin] RBAC: Authorizing `identity:get_project(project_id=402eb261fe224c1faa856398d15ee4e2)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:33.611527 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ab82b0-bf5a-4d05-88d4-66d139633c9d None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:33.614844 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 51/96] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Jul 26 13:32:33 2022] GET /identity/v3/projects/402eb261fe224c1faa856398d15ee4e2 => generated 312 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:33.622740 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b444ba37-eeaf-4e48-afa5-6f8a7697d73c None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:33.686288 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:33.773687 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b444ba37-eeaf-4e48-afa5-6f8a7697d73c None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:33.804726 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab8d228-e74c-4a3b-b0b1-2fffa73fb4fb None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:33.805166 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab8d228-e74c-4a3b-b0b1-2fffa73fb4fb None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:33.805582 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab8d228-e74c-4a3b-b0b1-2fffa73fb4fb None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:33.807108 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 52/97] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:32:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:32:33.822021 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34945a3-6cba-42b2-b645-9792beb05fe7 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:33.822462 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34945a3-6cba-42b2-b645-9792beb05fe7 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:33.822843 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34945a3-6cba-42b2-b645-9792beb05fe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:33.825110 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229b627a-60ce-4e73-853a-51dd4479e522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:33.826520 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b627a-60ce-4e73-853a-51dd4479e522 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:33.826951 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b627a-60ce-4e73-853a-51dd4479e522 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:33.827387 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b627a-60ce-4e73-853a-51dd4479e522 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:33.828334 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229b627a-60ce-4e73-853a-51dd4479e522 None admin] RBAC: Adding query filter params (role.id=759f389b221049c4b01faf3b984cd8d7, scope.project.id=402eb261fe224c1faa856398d15ee4e2, user.id=eac6099386504fcea65a5d57e8a6c1d2) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:32:33.828792 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229b627a-60ce-4e73-853a-51dd4479e522 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:33.830410 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229b627a-60ce-4e73-853a-51dd4479e522 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:33.843440 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 46/98] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Jul 26 13:32:33 2022] GET /identity/v3/role_assignments?user.id=eac6099386504fcea65a5d57e8a6c1d2&scope.project.id=402eb261fe224c1faa856398d15ee4e2&role.id=759f389b221049c4b01faf3b984cd8d7 => generated 259 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:33.849131 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-a34945a3-6cba-42b2-b645-9792beb05fe7 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:32:33.850429 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-a34945a3-6cba-42b2-b645-9792beb05fe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:32:33.938582 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c8fc73-1baf-4fb9-bde5-fcd4150289db None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:33.938903 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c8fc73-1baf-4fb9-bde5-fcd4150289db None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:33.939146 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c8fc73-1baf-4fb9-bde5-fcd4150289db None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:33.940547 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 47/99] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:32:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 13:32:33.948516 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a90f705-68ed-4403-95ff-fe5c92b868be None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:33.948860 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a90f705-68ed-4403-95ff-fe5c92b868be None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:33.949209 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a90f705-68ed-4403-95ff-fe5c92b868be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:33.956008 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 53/100] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:32:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 13:32:33.970838 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-9a90f705-68ed-4403-95ff-fe5c92b868be None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:32:33.972108 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-9a90f705-68ed-4403-95ff-fe5c92b868be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:32:33.990959 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34945a3-6cba-42b2-b645-9792beb05fe7 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:34.052558 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 48/101] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:32:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 13:32:34.076892 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a90f705-68ed-4403-95ff-fe5c92b868be None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:34.110891 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:34.129494 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:34.199604 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a90f705-68ed-4403-95ff-fe5c92b868be None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:34.227903 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34945a3-6cba-42b2-b645-9792beb05fe7 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:34.238583 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d527a7e3-a539-4b40-bd90-be975b639d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:34.239523 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d527a7e3-a539-4b40-bd90-be975b639d73 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:34.239951 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d527a7e3-a539-4b40-bd90-be975b639d73 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:34.240495 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d527a7e3-a539-4b40-bd90-be975b639d73 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:34.253368 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d527a7e3-a539-4b40-bd90-be975b639d73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:32:34.258797 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-d527a7e3-a539-4b40-bd90-be975b639d73 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 13:32:34.258797 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:32:34.258797 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:32:34.258797 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:32:34.258797 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:32:34.258797 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:32:34.258797 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:32:34.258797 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:32:34.258797 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:32:34.258797 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:32:34.258797 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:32:34.258797 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:32:34.258797 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:32:34.258797 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:32:34.258797 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:32:34.258797 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:32:34.259888 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:32:34.259888 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:32:34.259888 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:32:34.259888 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:32:34.259888 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:32:34.259888 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:32:34.259888 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:32:34.259888 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:32:34.259888 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:32:34.259888 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:32:34.259888 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:32:34.259888 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:32:34.259888 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:32:34.259888 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:32:34.259888 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:32:34.259888 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:32:34.260730 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:32:34.260730 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:32:34.260730 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:32:34.260730 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:32:34.260730 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:32:34.260730 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:32:34.260730 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:32:34.260730 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:32:34.260730 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:32:34.260730 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:32:34.260730 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 13:32:34.260730 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:32:34.262618 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 49/102] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Jul 26 13:32:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 13:32:34.268577 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d527a7e3-a539-4b40-bd90-be975b639d73 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:34.295591 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d6a6ac-062d-48d3-a69b-3a9b13cbf4a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:34.296241 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d6a6ac-062d-48d3-a69b-3a9b13cbf4a8 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:34.296581 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d6a6ac-062d-48d3-a69b-3a9b13cbf4a8 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:34.296954 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d6a6ac-062d-48d3-a69b-3a9b13cbf4a8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:34.299156 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d6a6ac-062d-48d3-a69b-3a9b13cbf4a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:34.301056 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d6a6ac-062d-48d3-a69b-3a9b13cbf4a8 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:34.302898 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 54/103] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:32:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:32:34.310576 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d6a6ac-062d-48d3-a69b-3a9b13cbf4a8 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:34.380099 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:34.384855 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d527a7e3-a539-4b40-bd90-be975b639d73 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:34.423365 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9729b6-e803-4f50-83ef-be0d80bd8b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:34.424262 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9729b6-e803-4f50-83ef-be0d80bd8b80 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:34.424726 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9729b6-e803-4f50-83ef-be0d80bd8b80 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:34.425149 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9729b6-e803-4f50-83ef-be0d80bd8b80 None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:34.426459 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9729b6-e803-4f50-83ef-be0d80bd8b80 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:32:34.426933 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9729b6-e803-4f50-83ef-be0d80bd8b80 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:34.428511 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9729b6-e803-4f50-83ef-be0d80bd8b80 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:34.439320 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 50/104] 10.222.0.58 () {56 vars in 1101 bytes} [Tue Jul 26 13:32:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:32:34.445281 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9729b6-e803-4f50-83ef-be0d80bd8b80 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:34.537600 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d6a6ac-062d-48d3-a69b-3a9b13cbf4a8 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:34.558057 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9729b6-e803-4f50-83ef-be0d80bd8b80 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:34.572667 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:34.601889 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b97711-395c-4018-b179-d04900d65ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:34.602835 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b97711-395c-4018-b179-d04900d65ea9 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:34.603263 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b97711-395c-4018-b179-d04900d65ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:34.603681 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b97711-395c-4018-b179-d04900d65ea9 None admin] PATH_INFO: `/v3/users/35d8fca391c145348cb1690f254308ae` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:34.609820 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acfc8d5-b515-48b7-8e22-3b3c65ae5eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:34.610468 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acfc8d5-b515-48b7-8e22-3b3c65ae5eb0 None admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:34.610752 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acfc8d5-b515-48b7-8e22-3b3c65ae5eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:34.611030 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acfc8d5-b515-48b7-8e22-3b3c65ae5eb0 None admin] PATH_INFO: `/v3/projects` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:34.611730 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acfc8d5-b515-48b7-8e22-3b3c65ae5eb0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:34.613190 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acfc8d5-b515-48b7-8e22-3b3c65ae5eb0 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:34.622830 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b97711-395c-4018-b179-d04900d65ea9 None admin] RBAC: Authorizing `identity:get_user(user_id=35d8fca391c145348cb1690f254308ae)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:34.624627 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b97711-395c-4018-b179-d04900d65ea9 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:34.626718 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 51/105] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Jul 26 13:32:34 2022] GET /identity/v3/users/35d8fca391c145348cb1690f254308ae => generated 289 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:32:34.632448 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b97711-395c-4018-b179-d04900d65ea9 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:34.659561 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:34.667071 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 55/106] 10.222.0.58 () {60 vars in 1141 bytes} [Tue Jul 26 13:32:34 2022] POST /identity/v3/projects => generated 302 bytes in 358 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 13:32:34.776918 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b97711-395c-4018-b179-d04900d65ea9 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:34.817566 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71278e20-8485-4940-93f8-f123b18ecee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:34.818546 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71278e20-8485-4940-93f8-f123b18ecee6 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:34.818990 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71278e20-8485-4940-93f8-f123b18ecee6 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:34.819412 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71278e20-8485-4940-93f8-f123b18ecee6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:34.898698 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71278e20-8485-4940-93f8-f123b18ecee6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:34.900394 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71278e20-8485-4940-93f8-f123b18ecee6 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:34.924586 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:35.034576 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 52/107] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Jul 26 13:32:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 26 13:32:35.046868 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acfc8d5-b515-48b7-8e22-3b3c65ae5eb0 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:35.177654 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acfc8d5-b515-48b7-8e22-3b3c65ae5eb0 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:35.221028 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33b47ee-f5b1-4d8f-9ffd-794c838e8a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:35.221721 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33b47ee-f5b1-4d8f-9ffd-794c838e8a95 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:35.222008 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33b47ee-f5b1-4d8f-9ffd-794c838e8a95 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:35.222290 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33b47ee-f5b1-4d8f-9ffd-794c838e8a95 None admin] PATH_INFO: `/v3/projects/afb9ca9466054e008e3b052866da363d` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:35.223484 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33b47ee-f5b1-4d8f-9ffd-794c838e8a95 None admin] RBAC: Authorizing `identity:get_project(project_id=afb9ca9466054e008e3b052866da363d)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:35.224942 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33b47ee-f5b1-4d8f-9ffd-794c838e8a95 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:35.226871 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 56/108] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Jul 26 13:32:35 2022] GET /identity/v3/projects/afb9ca9466054e008e3b052866da363d => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:35.232961 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71278e20-8485-4940-93f8-f123b18ecee6 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:35.387389 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71278e20-8485-4940-93f8-f123b18ecee6 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:35.413544 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:35.437991 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8be327f-e58c-4250-82fd-aec2752ab858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:35.438991 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8be327f-e58c-4250-82fd-aec2752ab858 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:35.439476 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8be327f-e58c-4250-82fd-aec2752ab858 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:35.439948 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8be327f-e58c-4250-82fd-aec2752ab858 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:35.440966 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8be327f-e58c-4250-82fd-aec2752ab858 None admin] RBAC: Adding query filter params (role.id=39ccb3797eb542529c79507664d04b53, scope.project.id=afb9ca9466054e008e3b052866da363d, user.id=35d8fca391c145348cb1690f254308ae) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:32:35.441499 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8be327f-e58c-4250-82fd-aec2752ab858 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:35.443183 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8be327f-e58c-4250-82fd-aec2752ab858 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:35.457506 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 53/109] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Jul 26 13:32:35 2022] GET /identity/v3/role_assignments?user.id=35d8fca391c145348cb1690f254308ae&scope.project.id=afb9ca9466054e008e3b052866da363d&role.id=39ccb3797eb542529c79507664d04b53 => generated 259 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:35.682674 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345423fd-c814-4c99-bafc-66d67a82adf9 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:35.683014 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345423fd-c814-4c99-bafc-66d67a82adf9 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:35.683230 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345423fd-c814-4c99-bafc-66d67a82adf9 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:35.684769 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 57/110] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:32:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:32:35.690019 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c43de4-d59b-43ce-93c2-efd8ec28a6ab None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:35.690538 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c43de4-d59b-43ce-93c2-efd8ec28a6ab None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:35.691031 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c43de4-d59b-43ce-93c2-efd8ec28a6ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:35.723023 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-c1c43de4-d59b-43ce-93c2-efd8ec28a6ab None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:32:35.724598 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-c1c43de4-d59b-43ce-93c2-efd8ec28a6ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:32:35.821965 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 54/111] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:32:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 13:32:35.856327 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345423fd-c814-4c99-bafc-66d67a82adf9 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:35.908907 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:35.985909 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345423fd-c814-4c99-bafc-66d67a82adf9 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:36.037849 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30338d87-e938-47cf-89b4-d72259f92fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:36.038530 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30338d87-e938-47cf-89b4-d72259f92fa8 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:36.038816 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30338d87-e938-47cf-89b4-d72259f92fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:36.039098 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30338d87-e938-47cf-89b4-d72259f92fa8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:36.041694 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c1c1ba-c939-44c8-ac0b-9a4a6b84ff43 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:36.042492 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c1c1ba-c939-44c8-ac0b-9a4a6b84ff43 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:36.042970 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c1c1ba-c939-44c8-ac0b-9a4a6b84ff43 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:36.049511 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 55/112] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:32:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:32:36.058474 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e892d688-9a23-493c-892c-d7c69a7bff15 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:36.059015 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e892d688-9a23-493c-892c-d7c69a7bff15 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:36.059492 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e892d688-9a23-493c-892c-d7c69a7bff15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:36.066500 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30338d87-e938-47cf-89b4-d72259f92fa8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:32:36.071579 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-30338d87-e938-47cf-89b4-d72259f92fa8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 13:32:36.071579 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:32:36.071579 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:32:36.071579 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:32:36.071579 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:32:36.071579 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:32:36.071579 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:32:36.071579 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:32:36.071579 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:32:36.071579 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:32:36.071579 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:32:36.071579 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:32:36.071579 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:32:36.071579 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:32:36.071579 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:32:36.071579 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:32:36.072461 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:32:36.072461 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:32:36.072461 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:32:36.072461 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:32:36.072461 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:32:36.072461 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:32:36.072461 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:32:36.072461 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:32:36.072461 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:32:36.072461 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:32:36.072461 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:32:36.072461 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:32:36.072461 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:32:36.072461 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:32:36.072461 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:32:36.072461 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:32:36.073230 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:32:36.073230 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:32:36.073230 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:32:36.073230 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:32:36.073230 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:32:36.073230 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:32:36.073230 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:32:36.073230 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:32:36.073230 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:32:36.073230 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:32:36.073230 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 13:32:36.073230 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:32:36.073230 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 58/113] 10.222.0.58 () {56 vars in 1114 bytes} [Tue Jul 26 13:32:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:32:36.081622 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30338d87-e938-47cf-89b4-d72259f92fa8 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:36.097805 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-e892d688-9a23-493c-892c-d7c69a7bff15 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:32:36.099383 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-e892d688-9a23-493c-892c-d7c69a7bff15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:32:36.272239 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 56/114] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:32:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 13:32:36.285914 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:36.293198 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30338d87-e938-47cf-89b4-d72259f92fa8 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:36.307350 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e892d688-9a23-493c-892c-d7c69a7bff15 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:36.355466 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ba0140-8e90-4220-8626-ccdfbc028e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:36.356119 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ba0140-8e90-4220-8626-ccdfbc028e6d None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:36.356403 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ba0140-8e90-4220-8626-ccdfbc028e6d None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:36.357224 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ba0140-8e90-4220-8626-ccdfbc028e6d None admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:36.358197 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ba0140-8e90-4220-8626-ccdfbc028e6d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:32:36.358326 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ba0140-8e90-4220-8626-ccdfbc028e6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:36.359683 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ba0140-8e90-4220-8626-ccdfbc028e6d None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:36.375893 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 59/115] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Jul 26 13:32:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:32:36.381913 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ba0140-8e90-4220-8626-ccdfbc028e6d None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:36.431836 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:36.528545 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ba0140-8e90-4220-8626-ccdfbc028e6d None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:36.608632 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ded1263-1c6d-488b-9bf9-713855b6616b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:36.609315 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ded1263-1c6d-488b-9bf9-713855b6616b None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:36.609635 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ded1263-1c6d-488b-9bf9-713855b6616b None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:36.609912 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ded1263-1c6d-488b-9bf9-713855b6616b None admin] PATH_INFO: `/v3/users/eac6099386504fcea65a5d57e8a6c1d2` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:36.612466 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ded1263-1c6d-488b-9bf9-713855b6616b None admin] RBAC: Authorizing `identity:get_user(user_id=eac6099386504fcea65a5d57e8a6c1d2)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:36.613986 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ded1263-1c6d-488b-9bf9-713855b6616b None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:36.615497 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e892d688-9a23-493c-892c-d7c69a7bff15 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:36.615626 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 60/116] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Jul 26 13:32:36 2022] GET /identity/v3/users/eac6099386504fcea65a5d57e8a6c1d2 => generated 275 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:32:36.623536 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ded1263-1c6d-488b-9bf9-713855b6616b None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:36.694549 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:36.699179 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca2d704-5419-4869-b647-5e1e95b2e8e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:36.699840 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca2d704-5419-4869-b647-5e1e95b2e8e4 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:36.700208 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca2d704-5419-4869-b647-5e1e95b2e8e4 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:36.700639 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca2d704-5419-4869-b647-5e1e95b2e8e4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:36.734159 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca2d704-5419-4869-b647-5e1e95b2e8e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:32:36.739231 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-1ca2d704-5419-4869-b647-5e1e95b2e8e4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 13:32:36.739231 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:32:36.739231 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:32:36.739231 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:32:36.739231 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:32:36.739231 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:32:36.739231 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:32:36.739231 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:32:36.739231 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:32:36.739231 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:32:36.739231 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:32:36.739231 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:32:36.739231 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:32:36.739231 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:32:36.739231 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:32:36.739231 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:32:36.740231 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:32:36.740231 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:32:36.740231 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:32:36.740231 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:32:36.740231 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:32:36.740231 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:32:36.740231 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:32:36.740231 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:32:36.740231 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:32:36.740231 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:32:36.740231 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:32:36.740231 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:32:36.740231 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:32:36.740231 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:32:36.740231 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:32:36.740231 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:32:36.740989 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:32:36.740989 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:32:36.740989 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:32:36.740989 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:32:36.740989 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:32:36.740989 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:32:36.740989 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:32:36.740989 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:32:36.740989 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:32:36.740989 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:32:36.740989 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 13:32:36.740989 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:32:36.740989 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 57/117] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Jul 26 13:32:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 434 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:32:36.742157 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2820f85-2898-4dfd-b96c-258eae86e513 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:36.742312 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2820f85-2898-4dfd-b96c-258eae86e513 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:36.742583 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2820f85-2898-4dfd-b96c-258eae86e513 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:36.744381 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 58/118] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:32:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:32:36.752962 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2820f85-2898-4dfd-b96c-258eae86e513 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:36.809553 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ded1263-1c6d-488b-9bf9-713855b6616b None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:36.877318 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba1f592-6c73-4093-8d54-c96662ba793f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:36.879648 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba1f592-6c73-4093-8d54-c96662ba793f None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:36.880390 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba1f592-6c73-4093-8d54-c96662ba793f None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:36.881348 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba1f592-6c73-4093-8d54-c96662ba793f None admin] PATH_INFO: `/v3/projects/afb9ca9466054e008e3b052866da363d` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:36.883762 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba1f592-6c73-4093-8d54-c96662ba793f None admin] RBAC: Authorizing `identity:get_project(project_id=afb9ca9466054e008e3b052866da363d)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:36.887310 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba1f592-6c73-4093-8d54-c96662ba793f None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:36.890874 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 61/119] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Jul 26 13:32:36 2022] GET /identity/v3/projects/afb9ca9466054e008e3b052866da363d => generated 298 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:32:36.893104 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ebdf60-1065-4a67-a0f8-453707d08bbd None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:36.893831 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ebdf60-1065-4a67-a0f8-453707d08bbd None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:36.894563 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ebdf60-1065-4a67-a0f8-453707d08bbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:36.928943 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-26ebdf60-1065-4a67-a0f8-453707d08bbd None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:32:36.931870 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-26ebdf60-1065-4a67-a0f8-453707d08bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:32:36.960652 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2820f85-2898-4dfd-b96c-258eae86e513 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:36.964617 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:37.007187 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6af84a-9a23-4e58-ac7a-073bb4441ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:37.008073 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6af84a-9a23-4e58-ac7a-073bb4441ec2 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:37.008442 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6af84a-9a23-4e58-ac7a-073bb4441ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:37.008796 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6af84a-9a23-4e58-ac7a-073bb4441ec2 None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:37.009994 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6af84a-9a23-4e58-ac7a-073bb4441ec2 None admin] RBAC: Adding query filter params (name=member) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:32:37.010387 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6af84a-9a23-4e58-ac7a-073bb4441ec2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:37.012211 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6af84a-9a23-4e58-ac7a-073bb4441ec2 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:37.019623 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 62/120] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:32:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 13:32:37.021599 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afbc302-dfe1-4569-888b-f57544988703 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:37.022141 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afbc302-dfe1-4569-888b-f57544988703 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:37.022924 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afbc302-dfe1-4569-888b-f57544988703 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:37.023558 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 59/121] 10.222.0.58 () {56 vars in 1101 bytes} [Tue Jul 26 13:32:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:37.024495 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6af84a-9a23-4e58-ac7a-073bb4441ec2 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:37.025540 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 63/122] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:32:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:32:37.029376 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afbc302-dfe1-4569-888b-f57544988703 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:37.073287 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:37.161767 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6af84a-9a23-4e58-ac7a-073bb4441ec2 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:37.186921 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afbc302-dfe1-4569-888b-f57544988703 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:37.207025 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3040a257-4e05-49b5-b38f-0a1aaeb2fedf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:37.208363 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3040a257-4e05-49b5-b38f-0a1aaeb2fedf None admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:37.209103 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3040a257-4e05-49b5-b38f-0a1aaeb2fedf None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:37.209696 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3040a257-4e05-49b5-b38f-0a1aaeb2fedf None admin] PATH_INFO: `/v3/projects/afb9ca9466054e008e3b052866da363d/users/eac6099386504fcea65a5d57e8a6c1d2/roles/5cb730ec786743c3bb8af83da90f9309` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:37.212901 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3040a257-4e05-49b5-b38f-0a1aaeb2fedf None admin] RBAC: Authorizing `identity:create_grant(project_id=afb9ca9466054e008e3b052866da363d, user_id=eac6099386504fcea65a5d57e8a6c1d2, role_id=5cb730ec786743c3bb8af83da90f9309)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:37.215498 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3040a257-4e05-49b5-b38f-0a1aaeb2fedf None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:37.230444 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef05ec5-f73c-45fc-a731-1b6a0be8e5bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:37.231063 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef05ec5-f73c-45fc-a731-1b6a0be8e5bc None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:37.231348 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef05ec5-f73c-45fc-a731-1b6a0be8e5bc None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:37.231739 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef05ec5-f73c-45fc-a731-1b6a0be8e5bc None admin] PATH_INFO: `/v3/users/eac6099386504fcea65a5d57e8a6c1d2` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:37.233178 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef05ec5-f73c-45fc-a731-1b6a0be8e5bc None admin] RBAC: Authorizing `identity:get_user(user_id=eac6099386504fcea65a5d57e8a6c1d2)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:37.234757 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef05ec5-f73c-45fc-a731-1b6a0be8e5bc None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:37.236772 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 64/123] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Jul 26 13:32:37 2022] GET /identity/v3/users/eac6099386504fcea65a5d57e8a6c1d2 => generated 275 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:32:37.242608 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e768c3-ceb7-46f5-8293-c47fcdf71b74 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:37.242931 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e768c3-ceb7-46f5-8293-c47fcdf71b74 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:37.243119 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 60/124] 10.222.0.58 () {58 vars in 1439 bytes} [Tue Jul 26 13:32:37 2022] PUT /identity/v3/projects/afb9ca9466054e008e3b052866da363d/users/eac6099386504fcea65a5d57e8a6c1d2/roles/5cb730ec786743c3bb8af83da90f9309 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:32:37.243225 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e768c3-ceb7-46f5-8293-c47fcdf71b74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:37.244441 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3040a257-4e05-49b5-b38f-0a1aaeb2fedf None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:37.266658 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-16e768c3-ceb7-46f5-8293-c47fcdf71b74 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:32:37.267989 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-16e768c3-ceb7-46f5-8293-c47fcdf71b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:32:37.318276 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:37.426168 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3040a257-4e05-49b5-b38f-0a1aaeb2fedf None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:37.428339 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 65/125] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:32:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 13:32:37.429493 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e768c3-ceb7-46f5-8293-c47fcdf71b74 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:37.471710 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:37.480445 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11db599a-86b7-4555-898d-fa77c9092231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:37.481280 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11db599a-86b7-4555-898d-fa77c9092231 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:37.481752 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11db599a-86b7-4555-898d-fa77c9092231 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:37.482373 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11db599a-86b7-4555-898d-fa77c9092231 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:37.494184 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11db599a-86b7-4555-898d-fa77c9092231 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:32:37.500381 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-11db599a-86b7-4555-898d-fa77c9092231 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 13:32:37.500381 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:32:37.500381 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:32:37.500381 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:32:37.500381 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:32:37.500381 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:32:37.500381 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:32:37.500381 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:32:37.500381 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:32:37.500381 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:32:37.500381 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:32:37.500381 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:32:37.500381 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:32:37.500381 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:32:37.500381 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:32:37.500381 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:32:37.501285 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:32:37.501285 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:32:37.501285 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:32:37.501285 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:32:37.501285 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:32:37.501285 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:32:37.501285 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:32:37.501285 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:32:37.501285 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:32:37.501285 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:32:37.501285 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:32:37.501285 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:32:37.501285 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:32:37.501285 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:32:37.501285 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:32:37.501285 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:32:37.502092 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:32:37.502092 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:32:37.502092 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:32:37.502092 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:32:37.502092 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:32:37.502092 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:32:37.502092 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:32:37.502092 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:32:37.502092 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:32:37.502092 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:32:37.502092 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 13:32:37.502092 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:32:37.502092 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 61/126] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Jul 26 13:32:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:32:37.502736 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11db599a-86b7-4555-898d-fa77c9092231 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:37.571999 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e768c3-ceb7-46f5-8293-c47fcdf71b74 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:37.598788 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:37.615774 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11db599a-86b7-4555-898d-fa77c9092231 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:37.616838 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a58eff3-a2c3-4bce-b8d2-efb0b9fa75af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:37.617912 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a58eff3-a2c3-4bce-b8d2-efb0b9fa75af None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:37.618348 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a58eff3-a2c3-4bce-b8d2-efb0b9fa75af None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:37.619736 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a58eff3-a2c3-4bce-b8d2-efb0b9fa75af None admin] PATH_INFO: `/v3/projects/afb9ca9466054e008e3b052866da363d` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:37.621283 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a58eff3-a2c3-4bce-b8d2-efb0b9fa75af None admin] RBAC: Authorizing `identity:get_project(project_id=afb9ca9466054e008e3b052866da363d)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:37.623479 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a58eff3-a2c3-4bce-b8d2-efb0b9fa75af None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:37.625990 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 66/127] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Jul 26 13:32:37 2022] GET /identity/v3/projects/afb9ca9466054e008e3b052866da363d => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:37.626974 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a58eff3-a2c3-4bce-b8d2-efb0b9fa75af None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:37.673980 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc34028-eecc-44cf-9f1b-486a409a2350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:37.674928 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc34028-eecc-44cf-9f1b-486a409a2350 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:37.675359 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc34028-eecc-44cf-9f1b-486a409a2350 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:37.676299 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc34028-eecc-44cf-9f1b-486a409a2350 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:37.710540 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc34028-eecc-44cf-9f1b-486a409a2350 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:32:37.720258 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-9bc34028-eecc-44cf-9f1b-486a409a2350 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 13:32:37.720258 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:32:37.720258 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:32:37.720258 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:32:37.720258 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:32:37.720258 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:32:37.720258 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:32:37.720258 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:32:37.720258 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:32:37.720258 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:32:37.720258 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:32:37.720258 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:32:37.720258 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:32:37.720258 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:32:37.720258 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:32:37.720258 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:32:37.721380 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:32:37.721380 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:32:37.721380 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:32:37.721380 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:32:37.721380 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:32:37.721380 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:32:37.721380 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:32:37.721380 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:32:37.721380 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:32:37.721380 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:32:37.721380 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:32:37.721380 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:32:37.721380 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:32:37.721380 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:32:37.721380 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:32:37.721380 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:32:37.722328 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:32:37.722328 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:32:37.722328 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:32:37.722328 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:32:37.722328 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:32:37.722328 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:32:37.722328 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:32:37.722328 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:32:37.722328 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:32:37.722328 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:32:37.722328 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 13:32:37.722328 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:32:37.722328 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 62/128] 10.222.0.58 () {56 vars in 1096 bytes} [Tue Jul 26 13:32:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:32:37.723042 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc34028-eecc-44cf-9f1b-486a409a2350 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:37.796179 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:37.818493 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a58eff3-a2c3-4bce-b8d2-efb0b9fa75af None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:37.897852 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235ff5f1-aa1e-4a02-a96d-8b6249ae6105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:37.898750 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235ff5f1-aa1e-4a02-a96d-8b6249ae6105 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:37.899040 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235ff5f1-aa1e-4a02-a96d-8b6249ae6105 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:37.899326 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235ff5f1-aa1e-4a02-a96d-8b6249ae6105 None admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:37.900062 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235ff5f1-aa1e-4a02-a96d-8b6249ae6105 None admin] RBAC: Adding query filter params (name=member) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:32:37.900371 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235ff5f1-aa1e-4a02-a96d-8b6249ae6105 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:37.902663 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235ff5f1-aa1e-4a02-a96d-8b6249ae6105 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:37.920941 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 67/129] 10.222.0.58 () {56 vars in 1101 bytes} [Tue Jul 26 13:32:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:37.921838 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235ff5f1-aa1e-4a02-a96d-8b6249ae6105 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:37.930389 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc34028-eecc-44cf-9f1b-486a409a2350 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:37.968115 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ebaf83-a482-4bbb-8ac4-571488f235d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:37.969106 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ebaf83-a482-4bbb-8ac4-571488f235d9 None admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:37.970350 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ebaf83-a482-4bbb-8ac4-571488f235d9 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:37.970640 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ebaf83-a482-4bbb-8ac4-571488f235d9 None admin] PATH_INFO: `/v3/projects/afb9ca9466054e008e3b052866da363d/users/eac6099386504fcea65a5d57e8a6c1d2/roles/759f389b221049c4b01faf3b984cd8d7` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:37.972869 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ebaf83-a482-4bbb-8ac4-571488f235d9 None admin] RBAC: Authorizing `identity:create_grant(project_id=afb9ca9466054e008e3b052866da363d, user_id=eac6099386504fcea65a5d57e8a6c1d2, role_id=759f389b221049c4b01faf3b984cd8d7)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:37.974386 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ebaf83-a482-4bbb-8ac4-571488f235d9 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:37.992708 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 63/130] 10.222.0.58 () {58 vars in 1439 bytes} [Tue Jul 26 13:32:37 2022] PUT /identity/v3/projects/afb9ca9466054e008e3b052866da363d/users/eac6099386504fcea65a5d57e8a6c1d2/roles/759f389b221049c4b01faf3b984cd8d7 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:32:37.993427 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ebaf83-a482-4bbb-8ac4-571488f235d9 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:38.009906 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:38.093772 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235ff5f1-aa1e-4a02-a96d-8b6249ae6105 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:38.179510 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b86035f-b886-4176-a254-28aae1e6e342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:38.180162 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b86035f-b886-4176-a254-28aae1e6e342 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:38.180507 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b86035f-b886-4176-a254-28aae1e6e342 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:38.180736 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b86035f-b886-4176-a254-28aae1e6e342 None admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:38.181770 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b86035f-b886-4176-a254-28aae1e6e342 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:32:38.182088 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b86035f-b886-4176-a254-28aae1e6e342 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:38.183538 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b86035f-b886-4176-a254-28aae1e6e342 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:38.201134 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 68/131] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Jul 26 13:32:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:38.206420 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ebaf83-a482-4bbb-8ac4-571488f235d9 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:38.206992 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f6cfa3-15c0-41b2-b6fb-d83d16163a37 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:38.207295 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f6cfa3-15c0-41b2-b6fb-d83d16163a37 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:38.207581 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f6cfa3-15c0-41b2-b6fb-d83d16163a37 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:38.208946 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 69/132] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:32:38 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:32:38.209700 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f6cfa3-15c0-41b2-b6fb-d83d16163a37 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:38.238149 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:38.254431 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d376f0f-2597-4bf3-a1be-029790530d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:38.255870 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d376f0f-2597-4bf3-a1be-029790530d92 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:38.256167 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d376f0f-2597-4bf3-a1be-029790530d92 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:38.256451 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d376f0f-2597-4bf3-a1be-029790530d92 None admin] PATH_INFO: `/v3/users/eac6099386504fcea65a5d57e8a6c1d2` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:38.258006 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d376f0f-2597-4bf3-a1be-029790530d92 None admin] RBAC: Authorizing `identity:get_user(user_id=eac6099386504fcea65a5d57e8a6c1d2)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:38.260155 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d376f0f-2597-4bf3-a1be-029790530d92 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:38.262075 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 64/133] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Jul 26 13:32:37 2022] GET /identity/v3/users/eac6099386504fcea65a5d57e8a6c1d2 => generated 275 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:38.263909 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81f2784-1945-4ec5-b709-dc0f3e36c3c1 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:38.264207 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81f2784-1945-4ec5-b709-dc0f3e36c3c1 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:38.264417 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81f2784-1945-4ec5-b709-dc0f3e36c3c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:38.285381 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-f81f2784-1945-4ec5-b709-dc0f3e36c3c1 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:32:38.286657 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-f81f2784-1945-4ec5-b709-dc0f3e36c3c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:32:38.370445 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 65/134] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:32:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 13:32:38.370639 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f6cfa3-15c0-41b2-b6fb-d83d16163a37 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:38.371865 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71857d3b-c895-403a-af56-d4684c2a7f80 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:38.372813 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71857d3b-c895-403a-af56-d4684c2a7f80 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:38.373107 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71857d3b-c895-403a-af56-d4684c2a7f80 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:38.374493 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 66/135] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:32:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:32:38.379331 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71857d3b-c895-403a-af56-d4684c2a7f80 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:38.383516 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:38.417867 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00f8a72-1fc3-47ea-8baf-262512aca749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:38.418573 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00f8a72-1fc3-47ea-8baf-262512aca749 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:38.418867 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00f8a72-1fc3-47ea-8baf-262512aca749 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:38.419183 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00f8a72-1fc3-47ea-8baf-262512aca749 None admin] PATH_INFO: `/v3/users/f1ff1f65366f4d20a56500bb6bc88c54` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:38.420720 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00f8a72-1fc3-47ea-8baf-262512aca749 None admin] RBAC: Authorizing `identity:get_user(user_id=f1ff1f65366f4d20a56500bb6bc88c54)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:38.422265 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00f8a72-1fc3-47ea-8baf-262512aca749 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:38.424143 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 70/136] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Jul 26 13:32:38 2022] GET /identity/v3/users/f1ff1f65366f4d20a56500bb6bc88c54 => generated 247 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:38.425335 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dbf1cc-59fa-4976-898f-7ed17a1dcc60 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:38.430300 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dbf1cc-59fa-4976-898f-7ed17a1dcc60 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:38.431011 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dbf1cc-59fa-4976-898f-7ed17a1dcc60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:38.452211 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-29dbf1cc-59fa-4976-898f-7ed17a1dcc60 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:32:38.453853 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-29dbf1cc-59fa-4976-898f-7ed17a1dcc60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:32:38.519344 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71857d3b-c895-403a-af56-d4684c2a7f80 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:38.548422 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 71/137] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:32:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 13:32:38.550403 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dbf1cc-59fa-4976-898f-7ed17a1dcc60 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:38.576132 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b4c445-bd74-44e5-976e-4d1e3383296c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:38.576798 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b4c445-bd74-44e5-976e-4d1e3383296c None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:38.577085 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b4c445-bd74-44e5-976e-4d1e3383296c None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:38.577362 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b4c445-bd74-44e5-976e-4d1e3383296c None admin] PATH_INFO: `/v3/projects/402eb261fe224c1faa856398d15ee4e2` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:38.579816 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b4c445-bd74-44e5-976e-4d1e3383296c None admin] RBAC: Authorizing `identity:get_project(project_id=402eb261fe224c1faa856398d15ee4e2)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:38.580633 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b4c445-bd74-44e5-976e-4d1e3383296c None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:38.582783 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 67/138] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Jul 26 13:32:38 2022] GET /identity/v3/projects/402eb261fe224c1faa856398d15ee4e2 => generated 312 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:38.583577 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b4c445-bd74-44e5-976e-4d1e3383296c None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:38.592927 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:38.665557 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:38.687837 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dbf1cc-59fa-4976-898f-7ed17a1dcc60 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:38.700877 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b4c445-bd74-44e5-976e-4d1e3383296c None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:38.733281 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472fe8d5-4a30-4545-a16f-1f3ab0866e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:38.734782 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472fe8d5-4a30-4545-a16f-1f3ab0866e4f None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:38.735468 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472fe8d5-4a30-4545-a16f-1f3ab0866e4f None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:38.736184 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472fe8d5-4a30-4545-a16f-1f3ab0866e4f None admin] PATH_INFO: `/v3/domains/default` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:38.738449 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472fe8d5-4a30-4545-a16f-1f3ab0866e4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:38.740854 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472fe8d5-4a30-4545-a16f-1f3ab0866e4f None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:38.741200 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a476ec7-dc81-4ab2-bda3-ba8df58c0167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:38.742059 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a476ec7-dc81-4ab2-bda3-ba8df58c0167 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:38.742346 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a476ec7-dc81-4ab2-bda3-ba8df58c0167 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:38.742626 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a476ec7-dc81-4ab2-bda3-ba8df58c0167 None admin] PATH_INFO: `/v3/projects/afb9ca9466054e008e3b052866da363d` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:38.743830 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a476ec7-dc81-4ab2-bda3-ba8df58c0167 None admin] RBAC: Authorizing `identity:get_project(project_id=afb9ca9466054e008e3b052866da363d)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:38.749520 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a476ec7-dc81-4ab2-bda3-ba8df58c0167 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:38.749520 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 68/139] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Jul 26 13:32:38 2022] GET /identity/v3/projects/afb9ca9466054e008e3b052866da363d => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:38.751252 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 72/140] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:32:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:38.752460 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472fe8d5-4a30-4545-a16f-1f3ab0866e4f None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:38.752933 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a476ec7-dc81-4ab2-bda3-ba8df58c0167 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:38.868195 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a476ec7-dc81-4ab2-bda3-ba8df58c0167 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:38.885913 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:38.905358 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f64cb6-92eb-4fe8-813e-b59b6f9ae9e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:38.906216 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f64cb6-92eb-4fe8-813e-b59b6f9ae9e6 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:38.906330 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f64cb6-92eb-4fe8-813e-b59b6f9ae9e6 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:38.906596 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f64cb6-92eb-4fe8-813e-b59b6f9ae9e6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:38.907091 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472fe8d5-4a30-4545-a16f-1f3ab0866e4f None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:38.915969 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:38.921751 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f64cb6-92eb-4fe8-813e-b59b6f9ae9e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:32:38.927220 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-11f64cb6-92eb-4fe8-813e-b59b6f9ae9e6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 13:32:38.927220 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:32:38.927220 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:32:38.927220 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:32:38.927220 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:32:38.927220 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:32:38.927220 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:32:38.927220 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:32:38.927220 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:32:38.927220 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:32:38.927220 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:32:38.927220 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:32:38.927220 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:32:38.927220 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:32:38.927220 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:32:38.927220 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:32:38.928156 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:32:38.928156 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:32:38.928156 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:32:38.928156 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:32:38.928156 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:32:38.928156 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:32:38.928156 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:32:38.928156 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:32:38.928156 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:32:38.928156 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:32:38.928156 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:32:38.928156 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:32:38.928156 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:32:38.928156 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:32:38.928156 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:32:38.928156 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:32:38.928940 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:32:38.928940 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:32:38.928940 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:32:38.928940 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:32:38.928940 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:32:38.928940 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:32:38.928940 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:32:38.928940 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:32:38.928940 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:32:38.928940 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:32:38.928940 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 13:32:38.928940 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:32:38.930957 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 69/141] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Jul 26 13:32:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:32:38.931966 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f64cb6-92eb-4fe8-813e-b59b6f9ae9e6 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:38.956721 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6bd08f-0038-4623-b5d7-e23dfa075011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:38.957515 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6bd08f-0038-4623-b5d7-e23dfa075011 None admin] REQUEST_METHOD: `PUT` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:38.957797 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6bd08f-0038-4623-b5d7-e23dfa075011 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:38.958159 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6bd08f-0038-4623-b5d7-e23dfa075011 None admin] PATH_INFO: `/v3/projects/402eb261fe224c1faa856398d15ee4e2/users/eac6099386504fcea65a5d57e8a6c1d2/roles/759f389b221049c4b01faf3b984cd8d7` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:38.960217 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6bd08f-0038-4623-b5d7-e23dfa075011 None admin] RBAC: Authorizing `identity:create_grant(project_id=402eb261fe224c1faa856398d15ee4e2, user_id=eac6099386504fcea65a5d57e8a6c1d2, role_id=759f389b221049c4b01faf3b984cd8d7)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:38.961759 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6bd08f-0038-4623-b5d7-e23dfa075011 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:38.980050 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 73/142] 10.222.0.58 () {58 vars in 1439 bytes} [Tue Jul 26 13:32:38 2022] PUT /identity/v3/projects/402eb261fe224c1faa856398d15ee4e2/users/eac6099386504fcea65a5d57e8a6c1d2/roles/759f389b221049c4b01faf3b984cd8d7 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:32:38.981016 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6bd08f-0038-4623-b5d7-e23dfa075011 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:39.075794 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f64cb6-92eb-4fe8-813e-b59b6f9ae9e6 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:39.115607 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6bd08f-0038-4623-b5d7-e23dfa075011 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:39.135192 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8e4f31-b2b4-49a4-aba0-a3c5efda3573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:39.135192 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8e4f31-b2b4-49a4-aba0-a3c5efda3573 None admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:39.135192 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8e4f31-b2b4-49a4-aba0-a3c5efda3573 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:39.135594 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8e4f31-b2b4-49a4-aba0-a3c5efda3573 None admin] PATH_INFO: `/v3/projects/afb9ca9466054e008e3b052866da363d/users/f1ff1f65366f4d20a56500bb6bc88c54/roles/6e0201a9a6484f7d8b112b1eafff70bd` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:39.138808 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8e4f31-b2b4-49a4-aba0-a3c5efda3573 None admin] RBAC: Authorizing `identity:create_grant(project_id=afb9ca9466054e008e3b052866da363d, user_id=f1ff1f65366f4d20a56500bb6bc88c54, role_id=6e0201a9a6484f7d8b112b1eafff70bd)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:39.140896 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8e4f31-b2b4-49a4-aba0-a3c5efda3573 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:39.176976 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 70/143] 10.222.0.58 () {58 vars in 1439 bytes} [Tue Jul 26 13:32:38 2022] PUT /identity/v3/projects/afb9ca9466054e008e3b052866da363d/users/f1ff1f65366f4d20a56500bb6bc88c54/roles/6e0201a9a6484f7d8b112b1eafff70bd => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:32:39.181774 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8e4f31-b2b4-49a4-aba0-a3c5efda3573 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:39.187003 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d3f8b7-8c5e-4571-a2ee-0c75accc4b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:39.188094 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d3f8b7-8c5e-4571-a2ee-0c75accc4b5a None admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:39.188593 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d3f8b7-8c5e-4571-a2ee-0c75accc4b5a None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:39.189102 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d3f8b7-8c5e-4571-a2ee-0c75accc4b5a None admin] PATH_INFO: `/v3/users` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:39.190071 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d3f8b7-8c5e-4571-a2ee-0c75accc4b5a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:39.193639 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d3f8b7-8c5e-4571-a2ee-0c75accc4b5a None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:39.235216 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:39.250640 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 74/144] 10.222.0.58 () {60 vars in 1132 bytes} [Tue Jul 26 13:32:38 2022] POST /identity/v3/users => generated 283 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:32:39.329879 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:39.389035 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8e4f31-b2b4-49a4-aba0-a3c5efda3573 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:39.430345 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374e35c3-5568-410a-a6d6-24cb48c624e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:39.430992 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374e35c3-5568-410a-a6d6-24cb48c624e4 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:39.431293 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374e35c3-5568-410a-a6d6-24cb48c624e4 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:39.431565 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374e35c3-5568-410a-a6d6-24cb48c624e4 None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:39.432296 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374e35c3-5568-410a-a6d6-24cb48c624e4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:32:39.432598 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374e35c3-5568-410a-a6d6-24cb48c624e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:39.434028 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374e35c3-5568-410a-a6d6-24cb48c624e4 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:39.444142 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 71/145] 10.222.0.58 () {56 vars in 1101 bytes} [Tue Jul 26 13:32:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:39.450618 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d3f8b7-8c5e-4571-a2ee-0c75accc4b5a None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:39.624198 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d3f8b7-8c5e-4571-a2ee-0c75accc4b5a None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:39.677511 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ccf43d-f008-429c-8269-0d24aea248a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:39.678551 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ccf43d-f008-429c-8269-0d24aea248a7 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:39.679047 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ccf43d-f008-429c-8269-0d24aea248a7 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:39.679552 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ccf43d-f008-429c-8269-0d24aea248a7 None admin] PATH_INFO: `/v3/users/35d8fca391c145348cb1690f254308ae` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:39.681020 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ccf43d-f008-429c-8269-0d24aea248a7 None admin] RBAC: Authorizing `identity:get_user(user_id=35d8fca391c145348cb1690f254308ae)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:39.682738 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ccf43d-f008-429c-8269-0d24aea248a7 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:39.685317 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 75/146] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Jul 26 13:32:39 2022] GET /identity/v3/users/35d8fca391c145348cb1690f254308ae => generated 289 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:39.691165 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374e35c3-5568-410a-a6d6-24cb48c624e4 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:39.826688 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc447f0b-3d58-45df-8dbe-e7505641a556 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:39.827427 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc447f0b-3d58-45df-8dbe-e7505641a556 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:39.827925 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc447f0b-3d58-45df-8dbe-e7505641a556 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:39.830062 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 76/147] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:32:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:32:39.833497 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374e35c3-5568-410a-a6d6-24cb48c624e4 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:39.839185 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aebf9d-47cd-4ac6-9bb7-6655e9395544 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:39.839479 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aebf9d-47cd-4ac6-9bb7-6655e9395544 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:39.839745 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aebf9d-47cd-4ac6-9bb7-6655e9395544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:39.863067 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:39.879651 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225ad302-cf04-4d8e-8ebb-4cc7f75fab63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:39.880558 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225ad302-cf04-4d8e-8ebb-4cc7f75fab63 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:39.881047 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225ad302-cf04-4d8e-8ebb-4cc7f75fab63 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:39.881645 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225ad302-cf04-4d8e-8ebb-4cc7f75fab63 None admin] PATH_INFO: `/v3/projects/afb9ca9466054e008e3b052866da363d` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:39.883156 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225ad302-cf04-4d8e-8ebb-4cc7f75fab63 None admin] RBAC: Authorizing `identity:get_project(project_id=afb9ca9466054e008e3b052866da363d)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:39.884673 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225ad302-cf04-4d8e-8ebb-4cc7f75fab63 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:39.886417 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 72/148] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Jul 26 13:32:39 2022] GET /identity/v3/projects/afb9ca9466054e008e3b052866da363d => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:39.886905 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-a5aebf9d-47cd-4ac6-9bb7-6655e9395544 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:32:39.888810 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-a5aebf9d-47cd-4ac6-9bb7-6655e9395544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:32:39.894516 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225ad302-cf04-4d8e-8ebb-4cc7f75fab63 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:40.017512 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225ad302-cf04-4d8e-8ebb-4cc7f75fab63 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:40.017866 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:40.022511 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 77/149] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:32:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 13:32:40.061541 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b53d947-7f61-4b13-a501-783620e634e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:40.062687 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b53d947-7f61-4b13-a501-783620e634e3 None admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:40.063188 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b53d947-7f61-4b13-a501-783620e634e3 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:40.063690 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b53d947-7f61-4b13-a501-783620e634e3 None admin] PATH_INFO: `/v3/projects/afb9ca9466054e008e3b052866da363d/users/35d8fca391c145348cb1690f254308ae/roles/39ccb3797eb542529c79507664d04b53` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:40.067160 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b53d947-7f61-4b13-a501-783620e634e3 None admin] RBAC: Authorizing `identity:create_grant(project_id=afb9ca9466054e008e3b052866da363d, user_id=35d8fca391c145348cb1690f254308ae, role_id=39ccb3797eb542529c79507664d04b53)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:40.068785 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b53d947-7f61-4b13-a501-783620e634e3 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:40.069167 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aebf9d-47cd-4ac6-9bb7-6655e9395544 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:40.093708 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 73/150] 10.222.0.58 () {58 vars in 1439 bytes} [Tue Jul 26 13:32:39 2022] PUT /identity/v3/projects/afb9ca9466054e008e3b052866da363d/users/35d8fca391c145348cb1690f254308ae/roles/39ccb3797eb542529c79507664d04b53 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 13:32:40.220345 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aebf9d-47cd-4ac6-9bb7-6655e9395544 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:40.268865 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8ac069-6776-4954-8042-3d116cf500af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:40.269924 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8ac069-6776-4954-8042-3d116cf500af None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:40.270359 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8ac069-6776-4954-8042-3d116cf500af None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:40.270793 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8ac069-6776-4954-8042-3d116cf500af None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:40.284977 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8ac069-6776-4954-8042-3d116cf500af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:32:40.290472 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-0a8ac069-6776-4954-8042-3d116cf500af None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 13:32:40.290472 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:32:40.290472 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:32:40.290472 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:32:40.290472 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:32:40.290472 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:32:40.290472 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:32:40.290472 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:32:40.290472 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:32:40.290472 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:32:40.290472 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:32:40.290472 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:32:40.290472 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:32:40.290472 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:32:40.290472 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:32:40.290472 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:32:40.291547 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:32:40.291547 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:32:40.291547 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:32:40.291547 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:32:40.291547 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:32:40.291547 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:32:40.291547 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:32:40.291547 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:32:40.291547 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:32:40.291547 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:32:40.291547 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:32:40.291547 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:32:40.291547 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:32:40.291547 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:32:40.291547 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:32:40.291547 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:32:40.292493 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:32:40.292493 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:32:40.292493 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:32:40.292493 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:32:40.292493 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:32:40.292493 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:32:40.292493 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:32:40.292493 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:32:40.292493 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:32:40.292493 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:32:40.292493 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 13:32:40.292493 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:32:40.295400 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 78/151] 10.222.0.58 () {56 vars in 1114 bytes} [Tue Jul 26 13:32:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:32:40.301732 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b53d947-7f61-4b13-a501-783620e634e3 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:40.413889 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:40.458034 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b53d947-7f61-4b13-a501-783620e634e3 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:40.501337 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23d0b39-7e44-444d-80ed-3997fc399d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:40.501918 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23d0b39-7e44-444d-80ed-3997fc399d99 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:40.502201 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23d0b39-7e44-444d-80ed-3997fc399d99 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:40.502479 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23d0b39-7e44-444d-80ed-3997fc399d99 None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:40.503218 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23d0b39-7e44-444d-80ed-3997fc399d99 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:32:40.503521 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23d0b39-7e44-444d-80ed-3997fc399d99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:40.505430 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23d0b39-7e44-444d-80ed-3997fc399d99 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:40.516117 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 74/152] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Jul 26 13:32:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:40.521748 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8ac069-6776-4954-8042-3d116cf500af None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:40.568978 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ff95fc-5855-43d6-aff9-89a1a25891e5 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:40.569364 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ff95fc-5855-43d6-aff9-89a1a25891e5 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:40.569814 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ff95fc-5855-43d6-aff9-89a1a25891e5 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:40.571198 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 75/153] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:32:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:32:40.583832 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e3d751-6a06-4068-a224-b64ea1d624b3 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:40.584213 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e3d751-6a06-4068-a224-b64ea1d624b3 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:40.584567 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e3d751-6a06-4068-a224-b64ea1d624b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:40.607780 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:40.627283 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-02e3d751-6a06-4068-a224-b64ea1d624b3 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:32:40.628994 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-02e3d751-6a06-4068-a224-b64ea1d624b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:32:40.658639 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8ac069-6776-4954-8042-3d116cf500af None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:40.702395 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c713987-4b83-4428-9c42-27c9b0a0def3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:40.703420 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c713987-4b83-4428-9c42-27c9b0a0def3 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:40.703839 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c713987-4b83-4428-9c42-27c9b0a0def3 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:40.704250 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c713987-4b83-4428-9c42-27c9b0a0def3 None admin] PATH_INFO: `/v3/users/eac6099386504fcea65a5d57e8a6c1d2` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:40.706578 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c713987-4b83-4428-9c42-27c9b0a0def3 None admin] RBAC: Authorizing `identity:get_user(user_id=eac6099386504fcea65a5d57e8a6c1d2)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:40.708347 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c713987-4b83-4428-9c42-27c9b0a0def3 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:40.713862 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 79/154] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Jul 26 13:32:40 2022] GET /identity/v3/users/eac6099386504fcea65a5d57e8a6c1d2 => generated 275 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:40.719746 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c713987-4b83-4428-9c42-27c9b0a0def3 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:40.743404 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 76/155] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:32:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 13:32:40.796552 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e3d751-6a06-4068-a224-b64ea1d624b3 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:40.855115 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:40.903908 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c713987-4b83-4428-9c42-27c9b0a0def3 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:40.922735 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e3d751-6a06-4068-a224-b64ea1d624b3 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:40.935805 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:40.960459 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f9d765-458f-4419-b8b7-e2018318de5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:40.961115 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f9d765-458f-4419-b8b7-e2018318de5c None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:40.961399 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f9d765-458f-4419-b8b7-e2018318de5c None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:40.961706 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f9d765-458f-4419-b8b7-e2018318de5c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:40.964862 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d708dec8-b709-460f-83cd-c35a748d3740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:40.965556 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d708dec8-b709-460f-83cd-c35a748d3740 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:40.966370 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d708dec8-b709-460f-83cd-c35a748d3740 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:40.966642 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d708dec8-b709-460f-83cd-c35a748d3740 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:40.979593 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d708dec8-b709-460f-83cd-c35a748d3740 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:32:40.985859 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-d708dec8-b709-460f-83cd-c35a748d3740 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 13:32:40.985859 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:32:40.985859 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:32:40.985859 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:32:40.985859 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:32:40.985859 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:32:40.985859 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:32:40.985859 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:32:40.985859 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:32:40.985859 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:32:40.985859 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:32:40.985859 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:32:40.985859 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:32:40.985859 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:32:40.985859 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:32:40.985859 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:32:40.986942 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:32:40.986942 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:32:40.986942 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:32:40.986942 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:32:40.986942 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:32:40.986942 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:32:40.986942 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:32:40.986942 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:32:40.986942 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:32:40.986942 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:32:40.986942 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:32:40.986942 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:32:40.986942 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:32:40.986942 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:32:40.986942 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:32:40.986942 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:32:40.987793 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:32:40.987793 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:32:40.987793 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:32:40.987793 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:32:40.987793 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:32:40.987793 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:32:40.987793 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:32:40.987793 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:32:40.987793 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:32:40.987793 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:32:40.987793 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 13:32:40.987793 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:32:40.987793 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 77/156] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Jul 26 13:32:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 13:32:40.994863 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d708dec8-b709-460f-83cd-c35a748d3740 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:41.104456 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f9d765-458f-4419-b8b7-e2018318de5c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:41.106127 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f9d765-458f-4419-b8b7-e2018318de5c None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:41.142292 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d708dec8-b709-460f-83cd-c35a748d3740 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:41.155537 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:41.188191 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8167fff7-7003-4e79-a713-c270328e4ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:41.188820 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8167fff7-7003-4e79-a713-c270328e4ee6 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:41.189100 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8167fff7-7003-4e79-a713-c270328e4ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:41.189367 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8167fff7-7003-4e79-a713-c270328e4ee6 None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:41.190144 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8167fff7-7003-4e79-a713-c270328e4ee6 None admin] RBAC: Adding query filter params (name=member) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:32:41.190446 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8167fff7-7003-4e79-a713-c270328e4ee6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:41.192189 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8167fff7-7003-4e79-a713-c270328e4ee6 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:41.202830 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 78/157] 10.222.0.58 () {56 vars in 1101 bytes} [Tue Jul 26 13:32:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:32:41.207900 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8167fff7-7003-4e79-a713-c270328e4ee6 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:41.283416 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 80/158] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Jul 26 13:32:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 564 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 26 13:32:41.289316 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f9d765-458f-4419-b8b7-e2018318de5c None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:41.327241 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8167fff7-7003-4e79-a713-c270328e4ee6 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:41.330606 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:41.369458 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa1065e-24d3-4887-8711-fa7ef22c9344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:41.370042 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa1065e-24d3-4887-8711-fa7ef22c9344 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:41.370328 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa1065e-24d3-4887-8711-fa7ef22c9344 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:41.370601 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa1065e-24d3-4887-8711-fa7ef22c9344 None admin] PATH_INFO: `/v3/users/eac6099386504fcea65a5d57e8a6c1d2` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:41.372859 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa1065e-24d3-4887-8711-fa7ef22c9344 None admin] RBAC: Authorizing `identity:get_user(user_id=eac6099386504fcea65a5d57e8a6c1d2)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:41.374487 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa1065e-24d3-4887-8711-fa7ef22c9344 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:41.376672 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 79/159] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Jul 26 13:32:41 2022] GET /identity/v3/users/eac6099386504fcea65a5d57e8a6c1d2 => generated 275 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:41.381750 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa1065e-24d3-4887-8711-fa7ef22c9344 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:41.442127 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:41.444699 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f9d765-458f-4419-b8b7-e2018318de5c None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:41.506620 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa1065e-24d3-4887-8711-fa7ef22c9344 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:41.523776 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d6be85-ab32-413f-b31b-6a48e28087f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:41.525138 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d6be85-ab32-413f-b31b-6a48e28087f3 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:41.525678 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d6be85-ab32-413f-b31b-6a48e28087f3 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:41.526251 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d6be85-ab32-413f-b31b-6a48e28087f3 None admin] PATH_INFO: `/v3/projects/afb9ca9466054e008e3b052866da363d` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:41.531028 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d6be85-ab32-413f-b31b-6a48e28087f3 None admin] RBAC: Authorizing `identity:get_project(project_id=afb9ca9466054e008e3b052866da363d)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:41.532826 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d6be85-ab32-413f-b31b-6a48e28087f3 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:41.535045 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 81/160] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Jul 26 13:32:41 2022] GET /identity/v3/projects/afb9ca9466054e008e3b052866da363d => generated 298 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:32:41.541026 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d6be85-ab32-413f-b31b-6a48e28087f3 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:41.545908 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e1a9b6-f50c-4e56-ad87-fa8550e80589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:41.546540 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e1a9b6-f50c-4e56-ad87-fa8550e80589 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:41.546822 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e1a9b6-f50c-4e56-ad87-fa8550e80589 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:41.547093 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e1a9b6-f50c-4e56-ad87-fa8550e80589 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:41.619332 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e1a9b6-f50c-4e56-ad87-fa8550e80589 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:41.621050 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e1a9b6-f50c-4e56-ad87-fa8550e80589 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:41.653244 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:41.733529 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 80/161] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Jul 26 13:32:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 26 13:32:41.740848 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e1a9b6-f50c-4e56-ad87-fa8550e80589 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:41.785208 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:41.803728 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d6be85-ab32-413f-b31b-6a48e28087f3 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:41.870237 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97a8a88-3af0-40f3-a2ad-6c3c09eadc7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:41.870862 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97a8a88-3af0-40f3-a2ad-6c3c09eadc7c None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:41.871196 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97a8a88-3af0-40f3-a2ad-6c3c09eadc7c None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:41.871477 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97a8a88-3af0-40f3-a2ad-6c3c09eadc7c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:41.873300 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97a8a88-3af0-40f3-a2ad-6c3c09eadc7c None admin] RBAC: Adding query filter params (role.id=5cb730ec786743c3bb8af83da90f9309, scope.project.id=afb9ca9466054e008e3b052866da363d, user.id=eac6099386504fcea65a5d57e8a6c1d2) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:32:41.873655 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97a8a88-3af0-40f3-a2ad-6c3c09eadc7c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:41.876269 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97a8a88-3af0-40f3-a2ad-6c3c09eadc7c None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:41.884953 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 82/162] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Jul 26 13:32:41 2022] GET /identity/v3/role_assignments?user.id=eac6099386504fcea65a5d57e8a6c1d2&scope.project.id=afb9ca9466054e008e3b052866da363d&role.id=5cb730ec786743c3bb8af83da90f9309 => generated 608 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:41.949107 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e1a9b6-f50c-4e56-ad87-fa8550e80589 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:41.988889 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:41.994692 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e965b00f-28dc-426e-a165-e50935146397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:41.995586 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e965b00f-28dc-426e-a165-e50935146397 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:41.996044 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e965b00f-28dc-426e-a165-e50935146397 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:41.996470 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e965b00f-28dc-426e-a165-e50935146397 None admin] PATH_INFO: `/v3/projects/afb9ca9466054e008e3b052866da363d` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:41.997907 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e965b00f-28dc-426e-a165-e50935146397 None admin] RBAC: Authorizing `identity:get_project(project_id=afb9ca9466054e008e3b052866da363d)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:41.999516 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e965b00f-28dc-426e-a165-e50935146397 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:42.001360 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 81/163] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Jul 26 13:32:41 2022] GET /identity/v3/projects/afb9ca9466054e008e3b052866da363d => generated 298 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:42.007977 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97a8a88-3af0-40f3-a2ad-6c3c09eadc7c None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:42.213645 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97a8a88-3af0-40f3-a2ad-6c3c09eadc7c None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:42.258639 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d35e5b-fe58-4527-815c-9d4393a658fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:42.259276 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d35e5b-fe58-4527-815c-9d4393a658fa None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:42.259556 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d35e5b-fe58-4527-815c-9d4393a658fa None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:42.259831 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d35e5b-fe58-4527-815c-9d4393a658fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:42.260635 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d35e5b-fe58-4527-815c-9d4393a658fa None admin] RBAC: Adding query filter params (role.id=759f389b221049c4b01faf3b984cd8d7, scope.project.id=afb9ca9466054e008e3b052866da363d, user.id=eac6099386504fcea65a5d57e8a6c1d2) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:32:42.260944 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d35e5b-fe58-4527-815c-9d4393a658fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:42.262450 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d35e5b-fe58-4527-815c-9d4393a658fa None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:42.274443 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 83/164] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Jul 26 13:32:42 2022] GET /identity/v3/role_assignments?user.id=eac6099386504fcea65a5d57e8a6c1d2&scope.project.id=afb9ca9466054e008e3b052866da363d&role.id=759f389b221049c4b01faf3b984cd8d7 => generated 608 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:42.703992 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fecc7e8-3027-4148-bf68-75c906885b46 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:42.705029 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fecc7e8-3027-4148-bf68-75c906885b46 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:42.705560 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fecc7e8-3027-4148-bf68-75c906885b46 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:42.707349 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 82/165] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:32:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:32:42.715366 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81efc4d7-2939-4de9-a4cb-379b48d14183 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:42.715946 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81efc4d7-2939-4de9-a4cb-379b48d14183 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:42.716147 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81efc4d7-2939-4de9-a4cb-379b48d14183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:42.748977 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-81efc4d7-2939-4de9-a4cb-379b48d14183 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:32:42.750838 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-81efc4d7-2939-4de9-a4cb-379b48d14183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:32:42.772411 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:42.838898 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 84/166] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:32:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 13:32:42.862836 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fecc7e8-3027-4148-bf68-75c906885b46 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:42.906552 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a3d80a-0b35-4d46-a2b1-b7a587ec5055 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:42.906855 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a3d80a-0b35-4d46-a2b1-b7a587ec5055 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:42.907104 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a3d80a-0b35-4d46-a2b1-b7a587ec5055 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:42.908594 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 85/167] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:32:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 13:32:42.914111 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35211798-2e09-4d31-82d3-e2023bdfa13d None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:42.914111 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35211798-2e09-4d31-82d3-e2023bdfa13d None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:42.914452 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35211798-2e09-4d31-82d3-e2023bdfa13d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:42.934847 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-35211798-2e09-4d31-82d3-e2023bdfa13d None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:32:42.936179 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-35211798-2e09-4d31-82d3-e2023bdfa13d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:32:42.984947 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fecc7e8-3027-4148-bf68-75c906885b46 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:43.034781 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05255fa2-3cda-4451-9073-30d89e96ce97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:43.036035 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05255fa2-3cda-4451-9073-30d89e96ce97 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:43.036746 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05255fa2-3cda-4451-9073-30d89e96ce97 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:43.037446 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05255fa2-3cda-4451-9073-30d89e96ce97 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:43.050874 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05255fa2-3cda-4451-9073-30d89e96ce97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:32:43.056448 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-05255fa2-3cda-4451-9073-30d89e96ce97 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 13:32:43.056448 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:32:43.056448 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:32:43.056448 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:32:43.056448 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:32:43.056448 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:32:43.056448 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:32:43.056448 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:32:43.056448 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:32:43.056448 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:32:43.056448 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:32:43.056448 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:32:43.056448 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:32:43.056448 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:32:43.056448 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:32:43.056448 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:32:43.057724 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:32:43.057724 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:32:43.057724 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:32:43.057724 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:32:43.057724 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:32:43.057724 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:32:43.057724 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:32:43.057724 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:32:43.057724 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:32:43.057724 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:32:43.057724 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:32:43.057724 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:32:43.057724 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:32:43.057724 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:32:43.057724 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:32:43.057724 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:32:43.058515 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:32:43.058515 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:32:43.058515 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:32:43.058515 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:32:43.058515 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:32:43.058515 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:32:43.058515 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:32:43.058515 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:32:43.058515 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:32:43.058515 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:32:43.058515 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 13:32:43.058515 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:32:43.059147 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 83/168] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Jul 26 13:32:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:32:43.065894 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05255fa2-3cda-4451-9073-30d89e96ce97 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:43.079212 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 86/169] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:32:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 13:32:43.116753 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35211798-2e09-4d31-82d3-e2023bdfa13d None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:43.153243 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:43.220809 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05255fa2-3cda-4451-9073-30d89e96ce97 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:43.277249 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fdc008-3082-4227-aa63-e51bf69ef877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:43.277910 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fdc008-3082-4227-aa63-e51bf69ef877 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:43.278239 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fdc008-3082-4227-aa63-e51bf69ef877 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:43.278520 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fdc008-3082-4227-aa63-e51bf69ef877 None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:43.279259 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fdc008-3082-4227-aa63-e51bf69ef877 None admin] RBAC: Adding query filter params (name=member) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:32:43.279567 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fdc008-3082-4227-aa63-e51bf69ef877 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:43.280963 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fdc008-3082-4227-aa63-e51bf69ef877 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:43.295285 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 84/170] 10.222.0.58 () {56 vars in 1101 bytes} [Tue Jul 26 13:32:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:43.301604 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fdc008-3082-4227-aa63-e51bf69ef877 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:43.311945 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:43.346594 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35211798-2e09-4d31-82d3-e2023bdfa13d None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:43.399303 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958c83bc-776a-4ab5-b590-d9574d067a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:43.399972 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958c83bc-776a-4ab5-b590-d9574d067a05 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:43.400251 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958c83bc-776a-4ab5-b590-d9574d067a05 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:43.400519 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958c83bc-776a-4ab5-b590-d9574d067a05 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:43.421436 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958c83bc-776a-4ab5-b590-d9574d067a05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:32:43.426689 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-958c83bc-776a-4ab5-b590-d9574d067a05 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 13:32:43.426689 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:32:43.426689 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:32:43.426689 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:32:43.426689 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:32:43.426689 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:32:43.426689 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:32:43.426689 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:32:43.426689 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:32:43.426689 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:32:43.426689 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:32:43.426689 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:32:43.426689 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:32:43.426689 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:32:43.426689 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:32:43.426689 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:32:43.427636 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:32:43.427636 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:32:43.427636 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:32:43.427636 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:32:43.427636 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:32:43.427636 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:32:43.427636 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:32:43.427636 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:32:43.427636 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:32:43.427636 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:32:43.427636 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:32:43.427636 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:32:43.427636 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:32:43.427636 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:32:43.427636 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:32:43.427636 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:32:43.428407 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:32:43.428407 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:32:43.428407 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:32:43.428407 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:32:43.428407 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:32:43.428407 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:32:43.428407 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:32:43.428407 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:32:43.428407 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:32:43.428407 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:32:43.428407 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 13:32:43.428407 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:32:43.429605 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 87/171] 10.222.0.58 () {56 vars in 1096 bytes} [Tue Jul 26 13:32:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:32:43.440289 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958c83bc-776a-4ab5-b590-d9574d067a05 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:43.534030 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fdc008-3082-4227-aa63-e51bf69ef877 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:43.592411 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:43.614195 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7547c095-a2cd-4537-80e4-fecf431cba33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:43.615014 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7547c095-a2cd-4537-80e4-fecf431cba33 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:43.615301 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7547c095-a2cd-4537-80e4-fecf431cba33 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:43.615575 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7547c095-a2cd-4537-80e4-fecf431cba33 None admin] PATH_INFO: `/v3/users/eac6099386504fcea65a5d57e8a6c1d2` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:43.617144 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7547c095-a2cd-4537-80e4-fecf431cba33 None admin] RBAC: Authorizing `identity:get_user(user_id=eac6099386504fcea65a5d57e8a6c1d2)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:43.618646 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7547c095-a2cd-4537-80e4-fecf431cba33 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:43.620610 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 85/172] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Jul 26 13:32:43 2022] GET /identity/v3/users/eac6099386504fcea65a5d57e8a6c1d2 => generated 275 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:32:43.629032 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7547c095-a2cd-4537-80e4-fecf431cba33 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:43.701583 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958c83bc-776a-4ab5-b590-d9574d067a05 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:43.794956 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f61a7fe-9aeb-4aa9-850b-8236f128bcfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:43.795563 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f61a7fe-9aeb-4aa9-850b-8236f128bcfe None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:43.795830 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f61a7fe-9aeb-4aa9-850b-8236f128bcfe None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:43.796076 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f61a7fe-9aeb-4aa9-850b-8236f128bcfe None admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:43.797369 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f61a7fe-9aeb-4aa9-850b-8236f128bcfe None admin] RBAC: Adding query filter params (name=admin) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:32:43.797715 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f61a7fe-9aeb-4aa9-850b-8236f128bcfe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:43.799657 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f61a7fe-9aeb-4aa9-850b-8236f128bcfe None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:43.817685 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 88/173] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Jul 26 13:32:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 382 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:43.823136 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f61a7fe-9aeb-4aa9-850b-8236f128bcfe None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:43.828192 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:43.858213 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7547c095-a2cd-4537-80e4-fecf431cba33 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:43.948206 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fc9823-e942-4f5e-a5dd-03d1ba796ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:43.948846 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fc9823-e942-4f5e-a5dd-03d1ba796ab3 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:43.949128 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fc9823-e942-4f5e-a5dd-03d1ba796ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:43.949398 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fc9823-e942-4f5e-a5dd-03d1ba796ab3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:44.019780 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f61a7fe-9aeb-4aa9-850b-8236f128bcfe None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:44.061876 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:44.079361 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fc9823-e942-4f5e-a5dd-03d1ba796ab3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:44.080850 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fc9823-e942-4f5e-a5dd-03d1ba796ab3 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:44.100027 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee67756-6456-4ffa-85b8-2af4b5f08b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:44.100698 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee67756-6456-4ffa-85b8-2af4b5f08b6d None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:44.100984 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee67756-6456-4ffa-85b8-2af4b5f08b6d None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:44.101264 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee67756-6456-4ffa-85b8-2af4b5f08b6d None admin] PATH_INFO: `/v3/users/f1ff1f65366f4d20a56500bb6bc88c54` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:44.102507 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee67756-6456-4ffa-85b8-2af4b5f08b6d None admin] RBAC: Authorizing `identity:get_user(user_id=f1ff1f65366f4d20a56500bb6bc88c54)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:44.104466 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee67756-6456-4ffa-85b8-2af4b5f08b6d None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:44.106429 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 89/174] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Jul 26 13:32:43 2022] GET /identity/v3/users/f1ff1f65366f4d20a56500bb6bc88c54 => generated 247 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:32:44.108208 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57706a6f-010e-40f8-b02d-f9edfb8236c4 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:44.108687 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57706a6f-010e-40f8-b02d-f9edfb8236c4 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:44.109110 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57706a6f-010e-40f8-b02d-f9edfb8236c4 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:44.110918 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 90/175] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:32:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:32:44.118668 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57706a6f-010e-40f8-b02d-f9edfb8236c4 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:44.211642 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 86/176] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Jul 26 13:32:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 583 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 26 13:32:44.213605 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a892b20c-2566-4c89-b810-8bea797adecb None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:44.214035 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a892b20c-2566-4c89-b810-8bea797adecb None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:44.214447 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a892b20c-2566-4c89-b810-8bea797adecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:44.234154 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-a892b20c-2566-4c89-b810-8bea797adecb None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:32:44.236501 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-a892b20c-2566-4c89-b810-8bea797adecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:32:44.282882 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:44.322534 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 87/177] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:32:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 13:32:44.323806 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a892b20c-2566-4c89-b810-8bea797adecb None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:44.324688 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57706a6f-010e-40f8-b02d-f9edfb8236c4 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:44.349630 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:44.374286 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe2df24-1185-4620-abe3-873f6ac2fe68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:44.375204 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe2df24-1185-4620-abe3-873f6ac2fe68 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:44.375636 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe2df24-1185-4620-abe3-873f6ac2fe68 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:44.376054 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe2df24-1185-4620-abe3-873f6ac2fe68 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:44.446080 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a892b20c-2566-4c89-b810-8bea797adecb None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:44.460323 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe2df24-1185-4620-abe3-873f6ac2fe68 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:44.462214 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe2df24-1185-4620-abe3-873f6ac2fe68 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:44.483547 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8141ee-b8d2-489f-a56b-2223fb8eeabd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:44.484385 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8141ee-b8d2-489f-a56b-2223fb8eeabd None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:44.484668 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8141ee-b8d2-489f-a56b-2223fb8eeabd None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:44.484943 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8141ee-b8d2-489f-a56b-2223fb8eeabd None admin] PATH_INFO: `/v3/projects/402eb261fe224c1faa856398d15ee4e2` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:44.486370 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8141ee-b8d2-489f-a56b-2223fb8eeabd None admin] RBAC: Authorizing `identity:get_project(project_id=402eb261fe224c1faa856398d15ee4e2)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:44.488117 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8141ee-b8d2-489f-a56b-2223fb8eeabd None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:44.490144 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 88/178] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Jul 26 13:32:44 2022] GET /identity/v3/projects/402eb261fe224c1faa856398d15ee4e2 => generated 312 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:44.491545 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3ce035-3463-420c-bd56-538784208222 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:44.492003 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3ce035-3463-420c-bd56-538784208222 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:44.492453 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3ce035-3463-420c-bd56-538784208222 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:44.494053 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 89/179] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:32:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:32:44.494961 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3ce035-3463-420c-bd56-538784208222 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:44.554685 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:44.593325 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 91/180] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Jul 26 13:32:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 13:32:44.598305 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:44.599898 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d85bd9d-5958-4d82-a1de-2ae3883a5695 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:44.600192 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d85bd9d-5958-4d82-a1de-2ae3883a5695 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:44.600470 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d85bd9d-5958-4d82-a1de-2ae3883a5695 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:44.601830 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 92/181] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:32:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:32:44.602762 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d85bd9d-5958-4d82-a1de-2ae3883a5695 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:44.609013 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3ce035-3463-420c-bd56-538784208222 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:44.648361 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9275b51-2999-42b8-9173-4a3e06e67464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:44.649007 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9275b51-2999-42b8-9173-4a3e06e67464 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:44.649285 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9275b51-2999-42b8-9173-4a3e06e67464 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:44.649584 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9275b51-2999-42b8-9173-4a3e06e67464 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:44.661891 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9275b51-2999-42b8-9173-4a3e06e67464 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:32:44.667098 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-e9275b51-2999-42b8-9173-4a3e06e67464 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 13:32:44.667098 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:32:44.667098 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:32:44.667098 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:32:44.667098 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:32:44.667098 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:32:44.667098 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:32:44.667098 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:32:44.667098 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:32:44.667098 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:32:44.667098 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:32:44.667098 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:32:44.667098 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:32:44.667098 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:32:44.667098 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:32:44.667098 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:32:44.668159 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:32:44.668159 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:32:44.668159 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:32:44.668159 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:32:44.668159 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:32:44.668159 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:32:44.668159 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:32:44.668159 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:32:44.668159 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:32:44.668159 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:32:44.668159 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:32:44.668159 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:32:44.668159 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:32:44.668159 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:32:44.668159 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:32:44.668159 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:32:44.669136 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:32:44.669136 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:32:44.669136 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:32:44.669136 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:32:44.669136 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:32:44.669136 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:32:44.669136 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:32:44.669136 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:32:44.669136 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:32:44.669136 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:32:44.669136 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 13:32:44.669136 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:32:44.676035 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 90/182] 10.222.0.58 () {56 vars in 1096 bytes} [Tue Jul 26 13:32:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:32:44.677608 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8f343a-d9ea-4208-9943-8b53159d0bbf None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:44.678038 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8f343a-d9ea-4208-9943-8b53159d0bbf None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:44.678495 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8f343a-d9ea-4208-9943-8b53159d0bbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:44.697367 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-ed8f343a-d9ea-4208-9943-8b53159d0bbf None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:32:44.698857 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-ed8f343a-d9ea-4208-9943-8b53159d0bbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:32:44.720340 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d85bd9d-5958-4d82-a1de-2ae3883a5695 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:44.757920 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2231a6-3221-4a3a-b33a-88a701d19c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:44.758552 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2231a6-3221-4a3a-b33a-88a701d19c36 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:44.758839 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2231a6-3221-4a3a-b33a-88a701d19c36 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:44.759121 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2231a6-3221-4a3a-b33a-88a701d19c36 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:44.759923 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2231a6-3221-4a3a-b33a-88a701d19c36 None admin] RBAC: Adding query filter params (role.id=759f389b221049c4b01faf3b984cd8d7, scope.project.id=402eb261fe224c1faa856398d15ee4e2, user.id=eac6099386504fcea65a5d57e8a6c1d2) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:32:44.760236 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2231a6-3221-4a3a-b33a-88a701d19c36 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:44.761716 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2231a6-3221-4a3a-b33a-88a701d19c36 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:44.777521 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 93/183] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Jul 26 13:32:44 2022] GET /identity/v3/role_assignments?user.id=eac6099386504fcea65a5d57e8a6c1d2&scope.project.id=402eb261fe224c1faa856398d15ee4e2&role.id=759f389b221049c4b01faf3b984cd8d7 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:44.777521 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2231a6-3221-4a3a-b33a-88a701d19c36 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:44.787196 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 91/184] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:32:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 13:32:44.788404 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65062f8-a995-44af-b4b7-8d64b4ced5b2 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:44.788764 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65062f8-a995-44af-b4b7-8d64b4ced5b2 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:44.789114 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65062f8-a995-44af-b4b7-8d64b4ced5b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:44.814760 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-b65062f8-a995-44af-b4b7-8d64b4ced5b2 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:32:44.815985 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-b65062f8-a995-44af-b4b7-8d64b4ced5b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:32:44.836667 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:44.907801 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 92/185] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:32:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 13:32:44.910261 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65062f8-a995-44af-b4b7-8d64b4ced5b2 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:44.927213 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2231a6-3221-4a3a-b33a-88a701d19c36 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:44.932463 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:44.971148 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1589db2-9c13-4584-9b6e-6c0e7a96dcd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:44.971810 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1589db2-9c13-4584-9b6e-6c0e7a96dcd0 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:44.972090 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1589db2-9c13-4584-9b6e-6c0e7a96dcd0 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:44.972361 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1589db2-9c13-4584-9b6e-6c0e7a96dcd0 None admin] PATH_INFO: `/v3/projects/afb9ca9466054e008e3b052866da363d` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:44.973583 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1589db2-9c13-4584-9b6e-6c0e7a96dcd0 None admin] RBAC: Authorizing `identity:get_project(project_id=afb9ca9466054e008e3b052866da363d)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:44.975029 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1589db2-9c13-4584-9b6e-6c0e7a96dcd0 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:44.976739 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 94/186] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Jul 26 13:32:44 2022] GET /identity/v3/projects/afb9ca9466054e008e3b052866da363d => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:44.978362 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1589db2-9c13-4584-9b6e-6c0e7a96dcd0 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:45.045544 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65062f8-a995-44af-b4b7-8d64b4ced5b2 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:45.081886 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30cf878-1432-40a2-bf9f-363ab7375fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:45.082516 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30cf878-1432-40a2-bf9f-363ab7375fab None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:45.082802 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30cf878-1432-40a2-bf9f-363ab7375fab None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:45.083082 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30cf878-1432-40a2-bf9f-363ab7375fab None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:45.083811 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30cf878-1432-40a2-bf9f-363ab7375fab None admin] RBAC: Adding query filter params (name=admin) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:32:45.084122 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30cf878-1432-40a2-bf9f-363ab7375fab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:45.085528 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30cf878-1432-40a2-bf9f-363ab7375fab None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:45.091607 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1589db2-9c13-4584-9b6e-6c0e7a96dcd0 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:45.093177 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:45.093995 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 93/187] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Jul 26 13:32:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:45.095627 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f190c0b9-77c8-47c1-bf6a-8e5ee7e45afb None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:45.095847 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f190c0b9-77c8-47c1-bf6a-8e5ee7e45afb None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:45.096123 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f190c0b9-77c8-47c1-bf6a-8e5ee7e45afb None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:45.097397 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 94/188] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:32:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:32:45.098146 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f190c0b9-77c8-47c1-bf6a-8e5ee7e45afb None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:45.132507 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870a09ac-9908-43c6-af53-474d993da990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:45.133159 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870a09ac-9908-43c6-af53-474d993da990 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:45.133443 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870a09ac-9908-43c6-af53-474d993da990 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:45.133750 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870a09ac-9908-43c6-af53-474d993da990 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:45.145204 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870a09ac-9908-43c6-af53-474d993da990 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:32:45.150708 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-870a09ac-9908-43c6-af53-474d993da990 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 13:32:45.150708 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:32:45.150708 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:32:45.150708 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:32:45.150708 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:32:45.150708 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:32:45.150708 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:32:45.150708 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:32:45.150708 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:32:45.150708 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:32:45.150708 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:32:45.150708 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:32:45.150708 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:32:45.150708 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:32:45.150708 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:32:45.150708 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:32:45.151705 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:32:45.151705 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:32:45.151705 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:32:45.151705 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:32:45.151705 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:32:45.151705 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:32:45.151705 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:32:45.151705 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:32:45.151705 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:32:45.151705 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:32:45.151705 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:32:45.151705 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:32:45.151705 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:32:45.151705 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:32:45.151705 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:32:45.151705 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:32:45.152496 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:32:45.152496 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:32:45.152496 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:32:45.152496 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:32:45.152496 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:32:45.152496 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:32:45.152496 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:32:45.152496 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:32:45.152496 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:32:45.152496 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:32:45.152496 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 13:32:45.152496 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:32:45.152496 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 95/189] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Jul 26 13:32:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:32:45.153313 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870a09ac-9908-43c6-af53-474d993da990 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:45.177434 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:45.213445 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f190c0b9-77c8-47c1-bf6a-8e5ee7e45afb None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:45.251013 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d374062-80a5-44cc-a1f8-3f441105d73d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:45.251998 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d374062-80a5-44cc-a1f8-3f441105d73d None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:45.252197 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d374062-80a5-44cc-a1f8-3f441105d73d None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:45.252626 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d374062-80a5-44cc-a1f8-3f441105d73d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:45.258188 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870a09ac-9908-43c6-af53-474d993da990 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:45.265061 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d374062-80a5-44cc-a1f8-3f441105d73d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:32:45.270584 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-3d374062-80a5-44cc-a1f8-3f441105d73d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 13:32:45.270584 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:32:45.270584 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:32:45.270584 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:32:45.270584 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:32:45.270584 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:32:45.270584 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:32:45.270584 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:32:45.270584 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:32:45.270584 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:32:45.270584 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:32:45.270584 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:32:45.270584 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:32:45.270584 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:32:45.270584 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:32:45.270584 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:32:45.271451 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:32:45.271451 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:32:45.271451 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:32:45.271451 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:32:45.271451 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:32:45.271451 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:32:45.271451 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:32:45.271451 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:32:45.271451 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:32:45.271451 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:32:45.271451 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:32:45.271451 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:32:45.271451 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:32:45.271451 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:32:45.271451 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:32:45.271451 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:32:45.272237 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:32:45.272237 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:32:45.272237 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:32:45.272237 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:32:45.272237 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:32:45.272237 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:32:45.272237 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:32:45.272237 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:32:45.272237 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:32:45.272237 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:32:45.272237 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 13:32:45.272237 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:32:45.272237 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 95/190] 10.222.0.58 () {56 vars in 1114 bytes} [Tue Jul 26 13:32:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:32:45.272868 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d374062-80a5-44cc-a1f8-3f441105d73d None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:45.301727 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb327579-aa99-40e6-879e-19d579623e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:45.302452 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb327579-aa99-40e6-879e-19d579623e88 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:45.302760 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb327579-aa99-40e6-879e-19d579623e88 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:45.303057 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb327579-aa99-40e6-879e-19d579623e88 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:45.304250 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb327579-aa99-40e6-879e-19d579623e88 None admin] RBAC: Adding query filter params (role.id=6e0201a9a6484f7d8b112b1eafff70bd, scope.project.id=afb9ca9466054e008e3b052866da363d, user.id=f1ff1f65366f4d20a56500bb6bc88c54) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:32:45.304579 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb327579-aa99-40e6-879e-19d579623e88 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:45.306485 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb327579-aa99-40e6-879e-19d579623e88 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:45.314812 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 96/191] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Jul 26 13:32:45 2022] GET /identity/v3/role_assignments?user.id=f1ff1f65366f4d20a56500bb6bc88c54&scope.project.id=afb9ca9466054e008e3b052866da363d&role.id=6e0201a9a6484f7d8b112b1eafff70bd => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:45.316127 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefcb098-0dd1-4bd9-990f-1c3bff2aa157 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:45.316420 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefcb098-0dd1-4bd9-990f-1c3bff2aa157 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:45.316700 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefcb098-0dd1-4bd9-990f-1c3bff2aa157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:45.336127 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-aefcb098-0dd1-4bd9-990f-1c3bff2aa157 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:32:45.337358 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-aefcb098-0dd1-4bd9-990f-1c3bff2aa157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:32:45.364683 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:45.411060 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d374062-80a5-44cc-a1f8-3f441105d73d None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:45.419947 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 97/192] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:32:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 13:32:45.430917 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefcb098-0dd1-4bd9-990f-1c3bff2aa157 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:45.447955 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:45.456002 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73af9fce-b4f9-45bf-b468-40d5648e77bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:45.456676 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af9fce-b4f9-45bf-b468-40d5648e77bb None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:45.456966 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af9fce-b4f9-45bf-b468-40d5648e77bb None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:45.457230 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af9fce-b4f9-45bf-b468-40d5648e77bb None admin] PATH_INFO: `/v3/users/b5a2469f69414f11896fc44a4f8536f1` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:45.472894 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73af9fce-b4f9-45bf-b468-40d5648e77bb None admin] RBAC: Authorizing `identity:get_user(user_id=b5a2469f69414f11896fc44a4f8536f1)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:45.474397 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73af9fce-b4f9-45bf-b468-40d5648e77bb None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:45.476116 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 96/193] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Jul 26 13:32:45 2022] GET /identity/v3/users/b5a2469f69414f11896fc44a4f8536f1 => generated 283 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:45.477095 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73af9fce-b4f9-45bf-b468-40d5648e77bb None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:45.567301 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefcb098-0dd1-4bd9-990f-1c3bff2aa157 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:45.599183 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73af9fce-b4f9-45bf-b468-40d5648e77bb None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:45.604752 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe01598-b362-430a-a853-489012dd4a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:45.605379 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe01598-b362-430a-a853-489012dd4a20 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:45.605691 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe01598-b362-430a-a853-489012dd4a20 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:45.606473 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe01598-b362-430a-a853-489012dd4a20 None admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:45.607214 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe01598-b362-430a-a853-489012dd4a20 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:32:45.607520 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe01598-b362-430a-a853-489012dd4a20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:45.608926 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe01598-b362-430a-a853-489012dd4a20 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:45.618858 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 98/194] 10.222.0.58 () {56 vars in 1101 bytes} [Tue Jul 26 13:32:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:45.619705 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe01598-b362-430a-a853-489012dd4a20 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:45.640941 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:45.644355 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93d2ab5-dd6a-4771-af4d-57867eed4c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:45.645067 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93d2ab5-dd6a-4771-af4d-57867eed4c09 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:45.645415 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93d2ab5-dd6a-4771-af4d-57867eed4c09 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:45.645804 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93d2ab5-dd6a-4771-af4d-57867eed4c09 None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:45.646537 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93d2ab5-dd6a-4771-af4d-57867eed4c09 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:32:45.646839 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93d2ab5-dd6a-4771-af4d-57867eed4c09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:45.648231 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93d2ab5-dd6a-4771-af4d-57867eed4c09 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:45.656723 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 97/195] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Jul 26 13:32:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:45.657598 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93d2ab5-dd6a-4771-af4d-57867eed4c09 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:45.714148 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:45.733380 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe01598-b362-430a-a853-489012dd4a20 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:45.769246 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93d2ab5-dd6a-4771-af4d-57867eed4c09 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:45.770634 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea52dc5-381d-4a97-8a45-c76cfa1cb792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:45.771285 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea52dc5-381d-4a97-8a45-c76cfa1cb792 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:45.771559 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea52dc5-381d-4a97-8a45-c76cfa1cb792 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:45.771960 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea52dc5-381d-4a97-8a45-c76cfa1cb792 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:45.773220 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea52dc5-381d-4a97-8a45-c76cfa1cb792 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:45.774787 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea52dc5-381d-4a97-8a45-c76cfa1cb792 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:45.776472 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 99/196] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:32:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:45.777247 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea52dc5-381d-4a97-8a45-c76cfa1cb792 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:45.821501 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04955472-8023-449e-926f-862d50d371c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:45.822147 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04955472-8023-449e-926f-862d50d371c0 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:45.822425 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04955472-8023-449e-926f-862d50d371c0 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:45.822697 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04955472-8023-449e-926f-862d50d371c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:45.869797 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:45.885702 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea52dc5-381d-4a97-8a45-c76cfa1cb792 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:45.896690 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04955472-8023-449e-926f-862d50d371c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:45.898223 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04955472-8023-449e-926f-862d50d371c0 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:45.922806 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170100a8-1840-44eb-a1f6-f08bcdd587ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:45.923464 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170100a8-1840-44eb-a1f6-f08bcdd587ef None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:45.923749 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170100a8-1840-44eb-a1f6-f08bcdd587ef None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:45.924037 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170100a8-1840-44eb-a1f6-f08bcdd587ef None admin] PATH_INFO: `/v3/users/35d8fca391c145348cb1690f254308ae` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:45.925635 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170100a8-1840-44eb-a1f6-f08bcdd587ef None admin] RBAC: Authorizing `identity:get_user(user_id=35d8fca391c145348cb1690f254308ae)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:45.927093 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170100a8-1840-44eb-a1f6-f08bcdd587ef None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:45.928739 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 100/197] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Jul 26 13:32:45 2022] GET /identity/v3/users/35d8fca391c145348cb1690f254308ae => generated 289 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:45.929639 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170100a8-1840-44eb-a1f6-f08bcdd587ef None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:45.951266 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:46.006912 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 98/198] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Jul 26 13:32:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 13:32:46.007664 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04955472-8023-449e-926f-862d50d371c0 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:46.046896 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170100a8-1840-44eb-a1f6-f08bcdd587ef None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:46.082235 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5619ac6-d59e-43ae-9802-12ef026f0b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:46.082895 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5619ac6-d59e-43ae-9802-12ef026f0b23 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:46.083177 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5619ac6-d59e-43ae-9802-12ef026f0b23 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:46.083465 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5619ac6-d59e-43ae-9802-12ef026f0b23 None admin] PATH_INFO: `/v3/users/b5a2469f69414f11896fc44a4f8536f1` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:46.084681 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5619ac6-d59e-43ae-9802-12ef026f0b23 None admin] RBAC: Authorizing `identity:get_user(user_id=b5a2469f69414f11896fc44a4f8536f1)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:46.086169 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5619ac6-d59e-43ae-9802-12ef026f0b23 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:46.087854 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 101/199] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Jul 26 13:32:45 2022] GET /identity/v3/users/b5a2469f69414f11896fc44a4f8536f1 => generated 283 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:46.088634 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5619ac6-d59e-43ae-9802-12ef026f0b23 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:46.118644 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04955472-8023-449e-926f-862d50d371c0 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:46.118893 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:46.154847 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aaf49b-8fd1-4429-820b-be475cee5807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:46.155470 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aaf49b-8fd1-4429-820b-be475cee5807 None admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:46.155751 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aaf49b-8fd1-4429-820b-be475cee5807 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:46.156020 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aaf49b-8fd1-4429-820b-be475cee5807 None admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:46.156703 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aaf49b-8fd1-4429-820b-be475cee5807 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:46.158181 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aaf49b-8fd1-4429-820b-be475cee5807 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:46.183324 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 99/200] 10.222.0.58 () {60 vars in 1132 bytes} [Tue Jul 26 13:32:46 2022] POST /identity/v3/users => generated 297 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:32:46.184207 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aaf49b-8fd1-4429-820b-be475cee5807 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:46.194397 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:46.222933 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5619ac6-d59e-43ae-9802-12ef026f0b23 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:46.281412 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce345e8-24af-4f28-9ce6-ca75f507aa52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:46.281412 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce345e8-24af-4f28-9ce6-ca75f507aa52 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:46.281412 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce345e8-24af-4f28-9ce6-ca75f507aa52 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:46.281412 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce345e8-24af-4f28-9ce6-ca75f507aa52 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:46.332470 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aaf49b-8fd1-4429-820b-be475cee5807 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:46.367709 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce345e8-24af-4f28-9ce6-ca75f507aa52 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:46.369197 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce345e8-24af-4f28-9ce6-ca75f507aa52 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:46.381088 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457d0c13-05d3-41ae-8b66-e2ed59f6a9fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:46.381997 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457d0c13-05d3-41ae-8b66-e2ed59f6a9fb None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:46.382127 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457d0c13-05d3-41ae-8b66-e2ed59f6a9fb None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:46.383206 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457d0c13-05d3-41ae-8b66-e2ed59f6a9fb None admin] PATH_INFO: `/v3/projects/6ad88492b683452a81ab7150c22b665e` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:46.384541 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457d0c13-05d3-41ae-8b66-e2ed59f6a9fb None admin] RBAC: Authorizing `identity:get_project(project_id=6ad88492b683452a81ab7150c22b665e)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:46.386550 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457d0c13-05d3-41ae-8b66-e2ed59f6a9fb None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:46.393388 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 100/201] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Jul 26 13:32:46 2022] GET /identity/v3/projects/6ad88492b683452a81ab7150c22b665e => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:46.394642 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457d0c13-05d3-41ae-8b66-e2ed59f6a9fb None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:46.403596 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:46.498932 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:46.514934 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 102/202] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Jul 26 13:32:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 13:32:46.515803 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce345e8-24af-4f28-9ce6-ca75f507aa52 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:46.537552 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457d0c13-05d3-41ae-8b66-e2ed59f6a9fb None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:46.592973 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55748148-eebf-4a33-bd73-a10bbbc2156b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:46.594630 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55748148-eebf-4a33-bd73-a10bbbc2156b None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:46.594630 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55748148-eebf-4a33-bd73-a10bbbc2156b None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:46.594630 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55748148-eebf-4a33-bd73-a10bbbc2156b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:46.642593 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce345e8-24af-4f28-9ce6-ca75f507aa52 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:46.672258 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:46.689398 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114ddeee-5a3e-4be5-ab91-41ad82f67cca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:46.690481 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ddeee-5a3e-4be5-ab91-41ad82f67cca None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:46.691069 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ddeee-5a3e-4be5-ab91-41ad82f67cca None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:46.691570 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ddeee-5a3e-4be5-ab91-41ad82f67cca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:46.692592 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114ddeee-5a3e-4be5-ab91-41ad82f67cca None admin] RBAC: Adding query filter params (role.id=6e0201a9a6484f7d8b112b1eafff70bd, scope.project.id=6ad88492b683452a81ab7150c22b665e, user.id=b5a2469f69414f11896fc44a4f8536f1) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:32:46.693128 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114ddeee-5a3e-4be5-ab91-41ad82f67cca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:46.694931 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114ddeee-5a3e-4be5-ab91-41ad82f67cca None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:46.698196 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55748148-eebf-4a33-bd73-a10bbbc2156b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:46.700360 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55748148-eebf-4a33-bd73-a10bbbc2156b None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:46.702593 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 103/203] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Jul 26 13:32:46 2022] GET /identity/v3/role_assignments?user.id=b5a2469f69414f11896fc44a4f8536f1&scope.project.id=6ad88492b683452a81ab7150c22b665e&role.id=6e0201a9a6484f7d8b112b1eafff70bd => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:46.703651 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114ddeee-5a3e-4be5-ab91-41ad82f67cca None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:46.850530 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114ddeee-5a3e-4be5-ab91-41ad82f67cca None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:46.877443 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:46.919670 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154c6ba7-e353-484d-b819-9e05f7113cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:46.920435 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154c6ba7-e353-484d-b819-9e05f7113cd3 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:46.920736 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154c6ba7-e353-484d-b819-9e05f7113cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:46.921001 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154c6ba7-e353-484d-b819-9e05f7113cd3 None admin] PATH_INFO: `/v3/projects/afb9ca9466054e008e3b052866da363d` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:46.922399 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154c6ba7-e353-484d-b819-9e05f7113cd3 None admin] RBAC: Authorizing `identity:get_project(project_id=afb9ca9466054e008e3b052866da363d)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:46.923870 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154c6ba7-e353-484d-b819-9e05f7113cd3 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:46.927430 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 104/204] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Jul 26 13:32:46 2022] GET /identity/v3/projects/afb9ca9466054e008e3b052866da363d => generated 298 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:46.931110 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 101/205] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Jul 26 13:32:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 537 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 13:32:46.934844 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154c6ba7-e353-484d-b819-9e05f7113cd3 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:46.938244 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55748148-eebf-4a33-bd73-a10bbbc2156b None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:47.021845 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:47.112264 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154c6ba7-e353-484d-b819-9e05f7113cd3 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:47.139696 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55748148-eebf-4a33-bd73-a10bbbc2156b None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:47.183058 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f8b0d2-f5cb-43a9-8048-eb1d88343b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:47.183691 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f8b0d2-f5cb-43a9-8048-eb1d88343b21 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:47.183966 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f8b0d2-f5cb-43a9-8048-eb1d88343b21 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:47.184936 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f8b0d2-f5cb-43a9-8048-eb1d88343b21 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:47.185608 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f8b0d2-f5cb-43a9-8048-eb1d88343b21 None admin] RBAC: Adding query filter params (role.id=39ccb3797eb542529c79507664d04b53, scope.project.id=afb9ca9466054e008e3b052866da363d, user.id=35d8fca391c145348cb1690f254308ae) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:32:47.185923 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f8b0d2-f5cb-43a9-8048-eb1d88343b21 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:47.187730 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f8b0d2-f5cb-43a9-8048-eb1d88343b21 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:47.194346 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 105/206] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Jul 26 13:32:46 2022] GET /identity/v3/role_assignments?user.id=35d8fca391c145348cb1690f254308ae&scope.project.id=afb9ca9466054e008e3b052866da363d&role.id=39ccb3797eb542529c79507664d04b53 => generated 608 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:47.218284 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5590d9-b3c2-4c78-89a7-241a8c81eb67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:47.218964 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5590d9-b3c2-4c78-89a7-241a8c81eb67 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:47.219258 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5590d9-b3c2-4c78-89a7-241a8c81eb67 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:47.219536 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5590d9-b3c2-4c78-89a7-241a8c81eb67 None admin] PATH_INFO: `/v3/projects/6ad88492b683452a81ab7150c22b665e` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:47.226565 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5590d9-b3c2-4c78-89a7-241a8c81eb67 None admin] RBAC: Authorizing `identity:get_project(project_id=6ad88492b683452a81ab7150c22b665e)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:47.228066 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5590d9-b3c2-4c78-89a7-241a8c81eb67 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:47.229910 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 102/207] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Jul 26 13:32:46 2022] GET /identity/v3/projects/6ad88492b683452a81ab7150c22b665e => generated 302 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:47.240446 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f8b0d2-f5cb-43a9-8048-eb1d88343b21 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:47.384528 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f8b0d2-f5cb-43a9-8048-eb1d88343b21 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:47.404267 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:47.423188 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3c6676-7ee4-4c5a-b32b-2e640ef8251f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:47.423840 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3c6676-7ee4-4c5a-b32b-2e640ef8251f None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:47.424150 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3c6676-7ee4-4c5a-b32b-2e640ef8251f None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:47.424399 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3c6676-7ee4-4c5a-b32b-2e640ef8251f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:47.425456 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3c6676-7ee4-4c5a-b32b-2e640ef8251f None admin] RBAC: Adding query filter params (role.id=5cb730ec786743c3bb8af83da90f9309, scope.project.id=6ad88492b683452a81ab7150c22b665e, user.id=b5a2469f69414f11896fc44a4f8536f1) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:32:47.425808 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3c6676-7ee4-4c5a-b32b-2e640ef8251f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:47.427266 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3c6676-7ee4-4c5a-b32b-2e640ef8251f None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:47.433727 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 106/208] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Jul 26 13:32:47 2022] GET /identity/v3/role_assignments?user.id=b5a2469f69414f11896fc44a4f8536f1&scope.project.id=6ad88492b683452a81ab7150c22b665e&role.id=5cb730ec786743c3bb8af83da90f9309 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:48.482093 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb556ea6-c752-4562-9a45-ff3c1d106cc0 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:48.482651 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb556ea6-c752-4562-9a45-ff3c1d106cc0 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:48.482651 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb556ea6-c752-4562-9a45-ff3c1d106cc0 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:48.484336 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 103/209] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:32:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:32:48.489939 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9d29ab-c4cb-438a-a02d-b7a97570f2ef None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:48.490484 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9d29ab-c4cb-438a-a02d-b7a97570f2ef None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:48.490484 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9d29ab-c4cb-438a-a02d-b7a97570f2ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:48.509923 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-0a9d29ab-c4cb-438a-a02d-b7a97570f2ef None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:32:48.511132 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-0a9d29ab-c4cb-438a-a02d-b7a97570f2ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:32:48.589612 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 107/210] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:32:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 13:32:48.607267 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb556ea6-c752-4562-9a45-ff3c1d106cc0 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:48.623095 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023f642a-34f2-464e-8265-57511e57a202 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:48.623463 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023f642a-34f2-464e-8265-57511e57a202 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:48.623744 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023f642a-34f2-464e-8265-57511e57a202 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:48.626277 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 108/211] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:32:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:32:48.631396 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a6b324-4c30-4f9f-b5b6-1693cc472b3e None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:48.631669 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a6b324-4c30-4f9f-b5b6-1693cc472b3e None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:48.632058 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a6b324-4c30-4f9f-b5b6-1693cc472b3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:48.651968 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-14a6b324-4c30-4f9f-b5b6-1693cc472b3e None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:32:48.653313 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-14a6b324-4c30-4f9f-b5b6-1693cc472b3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:32:48.664716 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:48.734731 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 109/212] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:32:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 13:32:48.741217 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb556ea6-c752-4562-9a45-ff3c1d106cc0 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:48.758256 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a6b324-4c30-4f9f-b5b6-1693cc472b3e None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:48.788297 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0089096-b9f7-4fad-92d2-b313e751983f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:48.789325 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0089096-b9f7-4fad-92d2-b313e751983f None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:48.789825 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0089096-b9f7-4fad-92d2-b313e751983f None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:48.790312 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0089096-b9f7-4fad-92d2-b313e751983f None admin] PATH_INFO: `/v3/roles/member` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:48.803507 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0089096-b9f7-4fad-92d2-b313e751983f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:32:48.805316 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:48.811262 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-f0089096-b9f7-4fad-92d2-b313e751983f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 13:32:48.811262 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:32:48.811262 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:32:48.811262 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:32:48.811262 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:32:48.811262 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:32:48.811262 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:32:48.811262 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:32:48.811262 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:32:48.811262 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:32:48.811262 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:32:48.811262 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:32:48.811262 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:32:48.811262 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:32:48.811262 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:32:48.811262 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:32:48.812179 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:32:48.812179 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:32:48.812179 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:32:48.812179 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:32:48.812179 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:32:48.812179 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:32:48.812179 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:32:48.812179 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:32:48.812179 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:32:48.812179 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:32:48.812179 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:32:48.812179 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:32:48.812179 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:32:48.812179 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:32:48.812179 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:32:48.812179 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:32:48.812954 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:32:48.812954 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:32:48.812954 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:32:48.812954 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:32:48.812954 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:32:48.812954 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:32:48.812954 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:32:48.812954 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:32:48.812954 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:32:48.812954 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:32:48.812954 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 13:32:48.812954 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:32:48.815729 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 104/213] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Jul 26 13:32:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:32:48.822481 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0089096-b9f7-4fad-92d2-b313e751983f None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:48.881667 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a6b324-4c30-4f9f-b5b6-1693cc472b3e None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:48.928179 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a725aa-112a-4fde-ba35-be1ed16cc837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:48.929676 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a725aa-112a-4fde-ba35-be1ed16cc837 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:48.930465 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a725aa-112a-4fde-ba35-be1ed16cc837 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:48.930934 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a725aa-112a-4fde-ba35-be1ed16cc837 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:48.932730 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a725aa-112a-4fde-ba35-be1ed16cc837 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:48.935259 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a725aa-112a-4fde-ba35-be1ed16cc837 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:48.942888 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 110/214] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:32:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:32:48.943397 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a725aa-112a-4fde-ba35-be1ed16cc837 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:48.958231 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0089096-b9f7-4fad-92d2-b313e751983f None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:48.960397 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:49.004034 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e817fb91-f56e-40f2-8bac-734496c3baef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:49.004872 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e817fb91-f56e-40f2-8bac-734496c3baef None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:49.005155 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e817fb91-f56e-40f2-8bac-734496c3baef None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:49.005431 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e817fb91-f56e-40f2-8bac-734496c3baef None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:49.006356 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e817fb91-f56e-40f2-8bac-734496c3baef None admin] RBAC: Adding query filter params (name=member) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:32:49.007038 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e817fb91-f56e-40f2-8bac-734496c3baef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:49.008429 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e817fb91-f56e-40f2-8bac-734496c3baef None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:49.021820 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 105/215] 10.222.0.58 () {56 vars in 1101 bytes} [Tue Jul 26 13:32:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:32:49.021820 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7c5411-c30e-4857-9eff-e3b16539d91b None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:49.021820 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7c5411-c30e-4857-9eff-e3b16539d91b None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:49.021820 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7c5411-c30e-4857-9eff-e3b16539d91b None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:49.021820 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 106/216] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:32:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:32:49.023402 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7c5411-c30e-4857-9eff-e3b16539d91b None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:49.074479 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a725aa-112a-4fde-ba35-be1ed16cc837 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:49.088705 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:49.114257 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5c1b59-418f-4d25-bf16-5b75e4e98819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:49.114889 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c1b59-418f-4d25-bf16-5b75e4e98819 None admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:49.115173 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c1b59-418f-4d25-bf16-5b75e4e98819 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:49.115454 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c1b59-418f-4d25-bf16-5b75e4e98819 None admin] PATH_INFO: `/v3/users` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:49.116168 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5c1b59-418f-4d25-bf16-5b75e4e98819 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:49.117641 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5c1b59-418f-4d25-bf16-5b75e4e98819 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:49.150808 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 111/217] 10.222.0.58 () {60 vars in 1132 bytes} [Tue Jul 26 13:32:48 2022] POST /identity/v3/users => generated 297 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:32:49.150808 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e1cd56-15b3-4136-8edb-11038e184dc0 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:49.150808 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e1cd56-15b3-4136-8edb-11038e184dc0 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:49.150808 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e1cd56-15b3-4136-8edb-11038e184dc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:49.167022 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-f6e1cd56-15b3-4136-8edb-11038e184dc0 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:32:49.169068 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-f6e1cd56-15b3-4136-8edb-11038e184dc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:32:49.174543 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7c5411-c30e-4857-9eff-e3b16539d91b None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:49.221699 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b92176-d330-4bef-8408-d7b022f4d29e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:49.222483 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b92176-d330-4bef-8408-d7b022f4d29e None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:49.222834 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b92176-d330-4bef-8408-d7b022f4d29e None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:49.223237 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b92176-d330-4bef-8408-d7b022f4d29e None admin] PATH_INFO: `/v3/users/7ca36ada68fc48929486a18a402e7200` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:49.241818 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b92176-d330-4bef-8408-d7b022f4d29e None admin] RBAC: Authorizing `identity:get_user(user_id=7ca36ada68fc48929486a18a402e7200)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:49.243586 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b92176-d330-4bef-8408-d7b022f4d29e None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:49.250864 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 107/218] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Jul 26 13:32:49 2022] GET /identity/v3/users/7ca36ada68fc48929486a18a402e7200 => generated 297 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:49.252243 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b92176-d330-4bef-8408-d7b022f4d29e None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:49.270535 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 112/219] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:32:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 13:32:49.285259 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:49.295322 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e1cd56-15b3-4136-8edb-11038e184dc0 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:49.388465 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b92176-d330-4bef-8408-d7b022f4d29e None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:49.421527 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:49.433657 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e1cd56-15b3-4136-8edb-11038e184dc0 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:49.442682 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cb966b-6893-47ed-9a1e-4ec6bdbd39d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:49.443361 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cb966b-6893-47ed-9a1e-4ec6bdbd39d6 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:49.443648 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cb966b-6893-47ed-9a1e-4ec6bdbd39d6 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:49.443922 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cb966b-6893-47ed-9a1e-4ec6bdbd39d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:49.491224 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19021c71-666f-4b37-9953-0b9385c39e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:49.491910 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19021c71-666f-4b37-9953-0b9385c39e73 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:49.492195 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19021c71-666f-4b37-9953-0b9385c39e73 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:49.492470 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19021c71-666f-4b37-9953-0b9385c39e73 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:49.510150 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19021c71-666f-4b37-9953-0b9385c39e73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:32:49.514516 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-19021c71-666f-4b37-9953-0b9385c39e73 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 13:32:49.514516 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:32:49.514516 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:32:49.514516 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:32:49.514516 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:32:49.514516 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:32:49.514516 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:32:49.514516 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:32:49.514516 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:32:49.514516 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:32:49.514516 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:32:49.514516 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:32:49.514516 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:32:49.514516 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:32:49.514516 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:32:49.514516 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:32:49.515485 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:32:49.515485 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:32:49.515485 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:32:49.515485 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:32:49.515485 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:32:49.515485 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:32:49.515485 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:32:49.515485 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:32:49.515485 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:32:49.515485 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:32:49.515485 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:32:49.515485 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:32:49.515485 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:32:49.515485 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:32:49.515485 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:32:49.515485 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:32:49.516461 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:32:49.516461 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:32:49.516461 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:32:49.516461 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:32:49.516461 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:32:49.516461 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:32:49.516461 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:32:49.516461 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:32:49.516461 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:32:49.516461 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:32:49.516461 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 13:32:49.516461 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:32:49.516461 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 113/220] 10.222.0.58 () {56 vars in 1096 bytes} [Tue Jul 26 13:32:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 13:32:49.525718 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19021c71-666f-4b37-9953-0b9385c39e73 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:49.558357 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cb966b-6893-47ed-9a1e-4ec6bdbd39d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:49.561239 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cb966b-6893-47ed-9a1e-4ec6bdbd39d6 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:49.591086 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:49.669496 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19021c71-666f-4b37-9953-0b9385c39e73 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:49.682621 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 108/221] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Jul 26 13:32:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 26 13:32:49.683788 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326bf4e5-e90e-424e-9eb4-c32b09ed4e47 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:49.684124 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326bf4e5-e90e-424e-9eb4-c32b09ed4e47 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:49.684398 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326bf4e5-e90e-424e-9eb4-c32b09ed4e47 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:49.685815 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 109/222] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:32:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:32:49.689952 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326bf4e5-e90e-424e-9eb4-c32b09ed4e47 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:49.712984 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ff0894-61e0-4610-b7f0-1d433a79f488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:49.712984 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ff0894-61e0-4610-b7f0-1d433a79f488 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:49.713266 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ff0894-61e0-4610-b7f0-1d433a79f488 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:49.713570 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ff0894-61e0-4610-b7f0-1d433a79f488 None admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:49.714432 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ff0894-61e0-4610-b7f0-1d433a79f488 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:32:49.714745 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ff0894-61e0-4610-b7f0-1d433a79f488 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:49.716668 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ff0894-61e0-4610-b7f0-1d433a79f488 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:49.730895 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 114/223] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Jul 26 13:32:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:32:49.732695 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4290863d-def4-4987-91ac-57ff9f09ac73 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:49.732989 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4290863d-def4-4987-91ac-57ff9f09ac73 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:49.733273 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4290863d-def4-4987-91ac-57ff9f09ac73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:49.756766 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-4290863d-def4-4987-91ac-57ff9f09ac73 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:32:49.758321 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-4290863d-def4-4987-91ac-57ff9f09ac73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:32:49.773200 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:49.814754 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326bf4e5-e90e-424e-9eb4-c32b09ed4e47 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:49.836059 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 115/224] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:32:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 13:32:49.842138 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:49.843979 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4290863d-def4-4987-91ac-57ff9f09ac73 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:49.855141 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cf762a-677d-40a3-896c-d7479827b2ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:49.855955 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cf762a-677d-40a3-896c-d7479827b2ce None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:49.856278 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cf762a-677d-40a3-896c-d7479827b2ce None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:49.856611 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cf762a-677d-40a3-896c-d7479827b2ce None admin] PATH_INFO: `/v3/projects/6ad88492b683452a81ab7150c22b665e` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:49.857926 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cf762a-677d-40a3-896c-d7479827b2ce None admin] RBAC: Authorizing `identity:get_project(project_id=6ad88492b683452a81ab7150c22b665e)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:49.859659 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cf762a-677d-40a3-896c-d7479827b2ce None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:49.861947 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 110/225] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Jul 26 13:32:49 2022] GET /identity/v3/projects/6ad88492b683452a81ab7150c22b665e => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:32:49.873512 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cf762a-677d-40a3-896c-d7479827b2ce None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:49.987725 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4290863d-def4-4987-91ac-57ff9f09ac73 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:49.995178 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cf762a-677d-40a3-896c-d7479827b2ce None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:50.028286 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a7124d-7491-4250-ad85-84a9fc14744f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:50.029323 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a7124d-7491-4250-ad85-84a9fc14744f None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:50.029961 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a7124d-7491-4250-ad85-84a9fc14744f None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:50.030529 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a7124d-7491-4250-ad85-84a9fc14744f None admin] PATH_INFO: `/v3/users/b5a2469f69414f11896fc44a4f8536f1` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:50.032001 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a7124d-7491-4250-ad85-84a9fc14744f None admin] RBAC: Authorizing `identity:get_user(user_id=b5a2469f69414f11896fc44a4f8536f1)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:50.033715 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a7124d-7491-4250-ad85-84a9fc14744f None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:50.035688 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 116/226] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Jul 26 13:32:49 2022] GET /identity/v3/users/b5a2469f69414f11896fc44a4f8536f1 => generated 283 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:50.036996 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a7124d-7491-4250-ad85-84a9fc14744f None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:50.043401 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54b9c9e-c612-4527-9a29-c2c4911f455a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:50.044333 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54b9c9e-c612-4527-9a29-c2c4911f455a None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:50.044622 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54b9c9e-c612-4527-9a29-c2c4911f455a None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:50.044937 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54b9c9e-c612-4527-9a29-c2c4911f455a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:50.046108 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54b9c9e-c612-4527-9a29-c2c4911f455a None admin] RBAC: Adding query filter params (role.id=759f389b221049c4b01faf3b984cd8d7, scope.project.id=6ad88492b683452a81ab7150c22b665e, user.id=7ca36ada68fc48929486a18a402e7200) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:32:50.046291 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54b9c9e-c612-4527-9a29-c2c4911f455a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:50.047722 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54b9c9e-c612-4527-9a29-c2c4911f455a None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:50.055343 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 111/227] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Jul 26 13:32:49 2022] GET /identity/v3/role_assignments?user.id=7ca36ada68fc48929486a18a402e7200&scope.project.id=6ad88492b683452a81ab7150c22b665e&role.id=759f389b221049c4b01faf3b984cd8d7 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:32:50.056355 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54b9c9e-c612-4527-9a29-c2c4911f455a None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:50.063647 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:50.125231 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:50.186149 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a7124d-7491-4250-ad85-84a9fc14744f None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:50.195295 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54b9c9e-c612-4527-9a29-c2c4911f455a None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:50.263961 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93fcdaf-3bc8-49df-b2e6-f752294754cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:50.264816 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93fcdaf-3bc8-49df-b2e6-f752294754cf None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:50.265105 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93fcdaf-3bc8-49df-b2e6-f752294754cf None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:50.267258 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93fcdaf-3bc8-49df-b2e6-f752294754cf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:50.273171 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405b8444-46bc-4d6d-b39c-075d36a01e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:50.273977 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405b8444-46bc-4d6d-b39c-075d36a01e28 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:50.274270 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405b8444-46bc-4d6d-b39c-075d36a01e28 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:50.274549 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405b8444-46bc-4d6d-b39c-075d36a01e28 None admin] PATH_INFO: `/v3/projects/6ad88492b683452a81ab7150c22b665e` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:50.276606 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405b8444-46bc-4d6d-b39c-075d36a01e28 None admin] RBAC: Authorizing `identity:get_project(project_id=6ad88492b683452a81ab7150c22b665e)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:50.279404 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405b8444-46bc-4d6d-b39c-075d36a01e28 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:50.281739 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 112/228] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Jul 26 13:32:50 2022] GET /identity/v3/projects/6ad88492b683452a81ab7150c22b665e => generated 302 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:50.285237 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93fcdaf-3bc8-49df-b2e6-f752294754cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:32:50.288514 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405b8444-46bc-4d6d-b39c-075d36a01e28 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:50.290896 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-b93fcdaf-3bc8-49df-b2e6-f752294754cf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 13:32:50.290896 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:32:50.290896 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:32:50.290896 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:32:50.290896 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:32:50.290896 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:32:50.290896 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:32:50.290896 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:32:50.290896 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:32:50.290896 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:32:50.290896 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:32:50.290896 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:32:50.290896 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:32:50.290896 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:32:50.290896 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:32:50.290896 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:32:50.291757 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:32:50.291757 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:32:50.291757 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:32:50.291757 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:32:50.291757 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:32:50.291757 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:32:50.291757 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:32:50.291757 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:32:50.291757 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:32:50.291757 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:32:50.291757 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:32:50.291757 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:32:50.291757 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:32:50.291757 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:32:50.291757 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:32:50.291757 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:32:50.292518 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:32:50.292518 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:32:50.292518 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:32:50.292518 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:32:50.292518 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:32:50.292518 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:32:50.292518 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:32:50.292518 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:32:50.292518 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:32:50.292518 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:32:50.292518 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 13:32:50.292518 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:32:50.295727 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 117/229] 10.222.0.58 () {56 vars in 1114 bytes} [Tue Jul 26 13:32:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:32:50.302077 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93fcdaf-3bc8-49df-b2e6-f752294754cf None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:50.400848 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:50.432511 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93fcdaf-3bc8-49df-b2e6-f752294754cf None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:50.476432 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405b8444-46bc-4d6d-b39c-075d36a01e28 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:50.504544 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab0020c-bff1-48ac-a200-bfb8b8d0f407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:50.505817 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab0020c-bff1-48ac-a200-bfb8b8d0f407 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:50.506237 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab0020c-bff1-48ac-a200-bfb8b8d0f407 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:50.507397 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab0020c-bff1-48ac-a200-bfb8b8d0f407 None admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:50.508287 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab0020c-bff1-48ac-a200-bfb8b8d0f407 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:32:50.509945 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab0020c-bff1-48ac-a200-bfb8b8d0f407 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:50.511359 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab0020c-bff1-48ac-a200-bfb8b8d0f407 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:50.519367 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:50.536028 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 118/230] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Jul 26 13:32:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:32:50.537025 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb9f5ee-24af-4b34-94d1-b8884c38f1f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:50.537156 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab0020c-bff1-48ac-a200-bfb8b8d0f407 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:50.538012 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb9f5ee-24af-4b34-94d1-b8884c38f1f5 None admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:50.538433 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb9f5ee-24af-4b34-94d1-b8884c38f1f5 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:50.538846 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb9f5ee-24af-4b34-94d1-b8884c38f1f5 None admin] PATH_INFO: `/v3/projects/6ad88492b683452a81ab7150c22b665e/users/b5a2469f69414f11896fc44a4f8536f1/roles/6e0201a9a6484f7d8b112b1eafff70bd` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:50.541011 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb9f5ee-24af-4b34-94d1-b8884c38f1f5 None admin] RBAC: Authorizing `identity:create_grant(project_id=6ad88492b683452a81ab7150c22b665e, user_id=b5a2469f69414f11896fc44a4f8536f1, role_id=6e0201a9a6484f7d8b112b1eafff70bd)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:50.542820 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb9f5ee-24af-4b34-94d1-b8884c38f1f5 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:50.572599 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 113/231] 10.222.0.58 () {58 vars in 1439 bytes} [Tue Jul 26 13:32:50 2022] PUT /identity/v3/projects/6ad88492b683452a81ab7150c22b665e/users/b5a2469f69414f11896fc44a4f8536f1/roles/6e0201a9a6484f7d8b112b1eafff70bd => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:32:50.737341 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab0020c-bff1-48ac-a200-bfb8b8d0f407 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:50.788361 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10da241-cee1-4f65-84d5-7e0c605954eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:50.789034 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10da241-cee1-4f65-84d5-7e0c605954eb None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:50.789317 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10da241-cee1-4f65-84d5-7e0c605954eb None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:50.789617 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10da241-cee1-4f65-84d5-7e0c605954eb None admin] PATH_INFO: `/v3/users/b5a2469f69414f11896fc44a4f8536f1` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:50.790851 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10da241-cee1-4f65-84d5-7e0c605954eb None admin] RBAC: Authorizing `identity:get_user(user_id=b5a2469f69414f11896fc44a4f8536f1)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:50.792294 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10da241-cee1-4f65-84d5-7e0c605954eb None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:50.794498 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 119/232] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Jul 26 13:32:50 2022] GET /identity/v3/users/b5a2469f69414f11896fc44a4f8536f1 => generated 283 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:32:50.800835 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb9f5ee-24af-4b34-94d1-b8884c38f1f5 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:50.951838 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb9f5ee-24af-4b34-94d1-b8884c38f1f5 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:50.991181 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02c845c-90f9-484c-a2c1-155be89c3510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:50.991809 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02c845c-90f9-484c-a2c1-155be89c3510 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:50.992095 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02c845c-90f9-484c-a2c1-155be89c3510 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:50.992370 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02c845c-90f9-484c-a2c1-155be89c3510 None admin] PATH_INFO: `/v3/projects/6ad88492b683452a81ab7150c22b665e` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:50.993644 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02c845c-90f9-484c-a2c1-155be89c3510 None admin] RBAC: Authorizing `identity:get_project(project_id=6ad88492b683452a81ab7150c22b665e)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:50.995115 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02c845c-90f9-484c-a2c1-155be89c3510 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:50.997226 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 114/233] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Jul 26 13:32:50 2022] GET /identity/v3/projects/6ad88492b683452a81ab7150c22b665e => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:51.002734 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10da241-cee1-4f65-84d5-7e0c605954eb None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:51.016499 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:51.134752 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10da241-cee1-4f65-84d5-7e0c605954eb None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:51.188296 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc922cdc-c302-4956-8341-49f7415ba10e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:51.188995 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc922cdc-c302-4956-8341-49f7415ba10e None admin] REQUEST_METHOD: `PUT` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:51.189282 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc922cdc-c302-4956-8341-49f7415ba10e None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:51.189589 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc922cdc-c302-4956-8341-49f7415ba10e None admin] PATH_INFO: `/v3/projects/6ad88492b683452a81ab7150c22b665e/users/b5a2469f69414f11896fc44a4f8536f1/roles/5cb730ec786743c3bb8af83da90f9309` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:51.192332 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc922cdc-c302-4956-8341-49f7415ba10e None admin] RBAC: Authorizing `identity:create_grant(project_id=6ad88492b683452a81ab7150c22b665e, user_id=b5a2469f69414f11896fc44a4f8536f1, role_id=5cb730ec786743c3bb8af83da90f9309)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:51.194062 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc922cdc-c302-4956-8341-49f7415ba10e None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:51.218523 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 120/234] 10.222.0.58 () {58 vars in 1439 bytes} [Tue Jul 26 13:32:51 2022] PUT /identity/v3/projects/6ad88492b683452a81ab7150c22b665e/users/b5a2469f69414f11896fc44a4f8536f1/roles/5cb730ec786743c3bb8af83da90f9309 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:32:51.966382 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b49ae1e-828a-4996-9e62-7206b6d1468b None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:51.967005 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b49ae1e-828a-4996-9e62-7206b6d1468b None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:51.967005 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b49ae1e-828a-4996-9e62-7206b6d1468b None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:51.968712 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 115/235] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:32:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:32:51.987247 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e81a2d9-e93f-4c76-9b48-7d0a420cebd4 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:51.987807 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e81a2d9-e93f-4c76-9b48-7d0a420cebd4 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:51.987807 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e81a2d9-e93f-4c76-9b48-7d0a420cebd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:52.009814 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e992f52-062a-499d-923e-f9bdddb01717 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:52.010119 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e992f52-062a-499d-923e-f9bdddb01717 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:52.010380 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e992f52-062a-499d-923e-f9bdddb01717 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:52.012564 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 116/236] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:32:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:32:52.014576 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-7e81a2d9-e93f-4c76-9b48-7d0a420cebd4 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:32:52.015799 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-7e81a2d9-e93f-4c76-9b48-7d0a420cebd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:32:52.017855 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce56469-b7ea-42c2-b87d-6bc9302e8434 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:52.018319 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce56469-b7ea-42c2-b87d-6bc9302e8434 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:52.018423 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce56469-b7ea-42c2-b87d-6bc9302e8434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:52.046802 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-dce56469-b7ea-42c2-b87d-6bc9302e8434 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:32:52.048022 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-dce56469-b7ea-42c2-b87d-6bc9302e8434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:32:52.144816 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:52.145899 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:52.160786 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 117/237] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:32:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 13:32:52.167043 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 121/238] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:32:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 13:32:52.194955 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce56469-b7ea-42c2-b87d-6bc9302e8434 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:52.214986 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e81a2d9-e93f-4c76-9b48-7d0a420cebd4 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:52.377597 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce56469-b7ea-42c2-b87d-6bc9302e8434 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:52.409320 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e81a2d9-e93f-4c76-9b48-7d0a420cebd4 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:52.441268 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac426eb-7d77-4bfa-9887-128a910bc72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:52.442274 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac426eb-7d77-4bfa-9887-128a910bc72e None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:52.442713 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac426eb-7d77-4bfa-9887-128a910bc72e None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:52.443148 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac426eb-7d77-4bfa-9887-128a910bc72e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:52.473450 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac426eb-7d77-4bfa-9887-128a910bc72e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:32:52.478709 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-dac426eb-7d77-4bfa-9887-128a910bc72e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 13:32:52.478709 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:32:52.478709 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:32:52.478709 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:32:52.478709 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:32:52.478709 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:32:52.478709 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:32:52.478709 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:32:52.478709 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:32:52.478709 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:32:52.478709 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:32:52.478709 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:32:52.478709 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:32:52.478709 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:32:52.478709 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:32:52.478709 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:32:52.479591 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:32:52.479591 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:32:52.479591 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:32:52.479591 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:32:52.479591 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:32:52.479591 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:32:52.479591 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:32:52.479591 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:32:52.479591 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:32:52.479591 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:32:52.479591 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:32:52.479591 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:32:52.479591 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:32:52.479591 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:32:52.479591 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:32:52.479591 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:32:52.480378 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:32:52.480378 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:32:52.480378 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:32:52.480378 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:32:52.480378 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:32:52.480378 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:32:52.480378 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:32:52.480378 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:32:52.480378 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:32:52.480378 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:32:52.480378 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 13:32:52.480378 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:32:52.485520 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 118/239] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Jul 26 13:32:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:32:52.488704 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac426eb-7d77-4bfa-9887-128a910bc72e None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:52.505805 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb8946a-18e9-41a8-a19f-3be6fae16418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:52.506527 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb8946a-18e9-41a8-a19f-3be6fae16418 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:52.506807 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb8946a-18e9-41a8-a19f-3be6fae16418 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:52.507075 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb8946a-18e9-41a8-a19f-3be6fae16418 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:52.510037 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb8946a-18e9-41a8-a19f-3be6fae16418 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:52.511484 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb8946a-18e9-41a8-a19f-3be6fae16418 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:52.515484 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 122/240] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:32:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:32:52.519346 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb8946a-18e9-41a8-a19f-3be6fae16418 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:52.578432 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:52.637787 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:52.676588 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac426eb-7d77-4bfa-9887-128a910bc72e None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:52.718192 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb8946a-18e9-41a8-a19f-3be6fae16418 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:52.726358 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8c3ac1-10c9-4384-a789-96636d23c059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:52.727793 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8c3ac1-10c9-4384-a789-96636d23c059 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:52.728464 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8c3ac1-10c9-4384-a789-96636d23c059 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:52.729239 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8c3ac1-10c9-4384-a789-96636d23c059 None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:52.731334 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8c3ac1-10c9-4384-a789-96636d23c059 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:32:52.732087 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8c3ac1-10c9-4384-a789-96636d23c059 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:52.734691 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8c3ac1-10c9-4384-a789-96636d23c059 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:52.751077 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 119/241] 10.222.0.58 () {56 vars in 1101 bytes} [Tue Jul 26 13:32:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:52.753791 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b73f3b-8eb2-4966-b8ee-256e4d2dcfae None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:52.754580 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b73f3b-8eb2-4966-b8ee-256e4d2dcfae None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:52.755352 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b73f3b-8eb2-4966-b8ee-256e4d2dcfae None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:52.765567 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 120/242] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:32:52 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:32:52.765567 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b73f3b-8eb2-4966-b8ee-256e4d2dcfae None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:52.767595 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee57ec1-c850-4fe3-bc01-3dd37a1632f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:52.768359 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee57ec1-c850-4fe3-bc01-3dd37a1632f9 None admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:52.768667 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee57ec1-c850-4fe3-bc01-3dd37a1632f9 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:52.768947 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee57ec1-c850-4fe3-bc01-3dd37a1632f9 None admin] PATH_INFO: `/v3/users` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:52.769782 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee57ec1-c850-4fe3-bc01-3dd37a1632f9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:52.771321 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee57ec1-c850-4fe3-bc01-3dd37a1632f9 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:52.797925 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 123/243] 10.222.0.58 () {60 vars in 1132 bytes} [Tue Jul 26 13:32:52 2022] POST /identity/v3/users => generated 293 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:32:52.799030 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e869c59-3794-4cc3-86ad-7f0978149e99 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:52.799500 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e869c59-3794-4cc3-86ad-7f0978149e99 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:52.800419 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e869c59-3794-4cc3-86ad-7f0978149e99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:52.822037 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-9e869c59-3794-4cc3-86ad-7f0978149e99 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:32:52.823418 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-9e869c59-3794-4cc3-86ad-7f0978149e99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:32:52.906631 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b73f3b-8eb2-4966-b8ee-256e4d2dcfae None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:52.910424 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:52.929536 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 124/244] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:32:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 13:32:52.963517 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cf9b3e-1437-4752-b7db-704087571c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:52.964571 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cf9b3e-1437-4752-b7db-704087571c18 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:52.965658 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cf9b3e-1437-4752-b7db-704087571c18 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:52.966151 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cf9b3e-1437-4752-b7db-704087571c18 None admin] PATH_INFO: `/v3/users/1e91c3827fba42fe81d50be2d6525528` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:52.981909 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e869c59-3794-4cc3-86ad-7f0978149e99 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:52.982913 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cf9b3e-1437-4752-b7db-704087571c18 None admin] RBAC: Authorizing `identity:get_user(user_id=1e91c3827fba42fe81d50be2d6525528)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:52.985033 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cf9b3e-1437-4752-b7db-704087571c18 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:52.987370 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 121/245] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Jul 26 13:32:52 2022] GET /identity/v3/users/1e91c3827fba42fe81d50be2d6525528 => generated 297 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:52.994547 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cf9b3e-1437-4752-b7db-704087571c18 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:53.060115 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:53.122632 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e869c59-3794-4cc3-86ad-7f0978149e99 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:53.175284 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383985e4-63c5-4ea2-a3cf-728fa0945d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:53.175898 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383985e4-63c5-4ea2-a3cf-728fa0945d41 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:53.176183 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383985e4-63c5-4ea2-a3cf-728fa0945d41 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:53.176595 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383985e4-63c5-4ea2-a3cf-728fa0945d41 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:53.187778 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cf9b3e-1437-4752-b7db-704087571c18 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:53.192442 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383985e4-63c5-4ea2-a3cf-728fa0945d41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:32:53.198786 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-383985e4-63c5-4ea2-a3cf-728fa0945d41 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 13:32:53.198786 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:32:53.198786 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:32:53.198786 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:32:53.198786 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:32:53.198786 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:32:53.198786 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:32:53.198786 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:32:53.198786 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:32:53.198786 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:32:53.198786 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:32:53.198786 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:32:53.198786 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:32:53.198786 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:32:53.198786 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:32:53.198786 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:32:53.200273 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:32:53.200273 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:32:53.200273 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:32:53.200273 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:32:53.200273 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:32:53.200273 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:32:53.200273 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:32:53.200273 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:32:53.200273 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:32:53.200273 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:32:53.200273 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:32:53.200273 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:32:53.200273 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:32:53.200273 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:32:53.200273 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:32:53.200273 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:32:53.201052 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:32:53.201052 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:32:53.201052 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:32:53.201052 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:32:53.201052 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:32:53.201052 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:32:53.201052 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:32:53.201052 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:32:53.201052 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:32:53.201052 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:32:53.201052 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 13:32:53.201052 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:32:53.201052 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 125/246] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Jul 26 13:32:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:32:53.207701 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383985e4-63c5-4ea2-a3cf-728fa0945d41 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:53.249645 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0414df-5df1-4b48-bbca-223c8bf3eec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:53.250287 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0414df-5df1-4b48-bbca-223c8bf3eec0 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:53.250568 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0414df-5df1-4b48-bbca-223c8bf3eec0 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:53.250849 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0414df-5df1-4b48-bbca-223c8bf3eec0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:53.288458 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:53.353329 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0414df-5df1-4b48-bbca-223c8bf3eec0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:53.355128 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0414df-5df1-4b48-bbca-223c8bf3eec0 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:53.355487 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383985e4-63c5-4ea2-a3cf-728fa0945d41 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:53.373831 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:53.418579 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac457fa-5ce2-4ca0-aeb5-5cdd4dfd4ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:53.419223 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac457fa-5ce2-4ca0-aeb5-5cdd4dfd4ae2 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:53.419506 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac457fa-5ce2-4ca0-aeb5-5cdd4dfd4ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:53.419780 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac457fa-5ce2-4ca0-aeb5-5cdd4dfd4ae2 None admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:53.420524 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac457fa-5ce2-4ca0-aeb5-5cdd4dfd4ae2 None admin] RBAC: Adding query filter params (name=member) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:32:53.420830 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac457fa-5ce2-4ca0-aeb5-5cdd4dfd4ae2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:53.422305 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac457fa-5ce2-4ca0-aeb5-5cdd4dfd4ae2 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:53.437289 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 126/247] 10.222.0.58 () {56 vars in 1101 bytes} [Tue Jul 26 13:32:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:53.438990 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142ee248-9538-42f3-baad-da1db374c76d None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:53.440007 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142ee248-9538-42f3-baad-da1db374c76d None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:53.440511 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142ee248-9538-42f3-baad-da1db374c76d None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:53.442525 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 127/248] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:32:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:32:53.448784 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142ee248-9538-42f3-baad-da1db374c76d None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:53.514278 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 122/249] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Jul 26 13:32:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 519 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 13:32:53.514796 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bc9cfc-66e4-42c6-83d6-df160096be5f None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:53.515149 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bc9cfc-66e4-42c6-83d6-df160096be5f None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:53.515427 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bc9cfc-66e4-42c6-83d6-df160096be5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:53.546994 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-c4bc9cfc-66e4-42c6-83d6-df160096be5f None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:32:53.549908 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-c4bc9cfc-66e4-42c6-83d6-df160096be5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:32:53.623702 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142ee248-9538-42f3-baad-da1db374c76d None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:53.647706 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:53.664529 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 123/250] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:32:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 13:32:53.665414 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bc9cfc-66e4-42c6-83d6-df160096be5f None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:53.721972 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe019386-3cfb-4c94-95ee-45fa893b2d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:53.723021 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe019386-3cfb-4c94-95ee-45fa893b2d4d None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:53.723564 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe019386-3cfb-4c94-95ee-45fa893b2d4d None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:53.724033 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe019386-3cfb-4c94-95ee-45fa893b2d4d None admin] PATH_INFO: `/v3/users/7ca36ada68fc48929486a18a402e7200` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:53.730333 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe019386-3cfb-4c94-95ee-45fa893b2d4d None admin] RBAC: Authorizing `identity:get_user(user_id=7ca36ada68fc48929486a18a402e7200)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:53.732441 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe019386-3cfb-4c94-95ee-45fa893b2d4d None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:53.734964 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 128/251] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Jul 26 13:32:53 2022] GET /identity/v3/users/7ca36ada68fc48929486a18a402e7200 => generated 297 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:53.735960 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe019386-3cfb-4c94-95ee-45fa893b2d4d None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:53.783418 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bc9cfc-66e4-42c6-83d6-df160096be5f None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:53.827653 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:53.835743 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d3823c-83b0-45c2-8020-601c6d28a04a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:53.837033 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d3823c-83b0-45c2-8020-601c6d28a04a None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:53.837951 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d3823c-83b0-45c2-8020-601c6d28a04a None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:53.838438 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d3823c-83b0-45c2-8020-601c6d28a04a None admin] PATH_INFO: `/v3/projects/6ad88492b683452a81ab7150c22b665e` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:53.841715 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d3823c-83b0-45c2-8020-601c6d28a04a None admin] RBAC: Authorizing `identity:get_project(project_id=6ad88492b683452a81ab7150c22b665e)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:53.843217 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d3823c-83b0-45c2-8020-601c6d28a04a None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:53.849702 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 124/252] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Jul 26 13:32:53 2022] GET /identity/v3/projects/6ad88492b683452a81ab7150c22b665e => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:53.849702 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d3823c-83b0-45c2-8020-601c6d28a04a None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:53.904189 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe019386-3cfb-4c94-95ee-45fa893b2d4d None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:53.937105 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:53.944334 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742ff214-c177-4297-b3f3-8ee976dedc7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:53.945112 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742ff214-c177-4297-b3f3-8ee976dedc7c None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:53.945507 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742ff214-c177-4297-b3f3-8ee976dedc7c None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:53.945886 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742ff214-c177-4297-b3f3-8ee976dedc7c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:53.960303 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742ff214-c177-4297-b3f3-8ee976dedc7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:32:53.966678 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-742ff214-c177-4297-b3f3-8ee976dedc7c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 13:32:53.966678 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:32:53.966678 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:32:53.966678 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:32:53.966678 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:32:53.966678 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:32:53.966678 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:32:53.966678 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:32:53.966678 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:32:53.966678 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:32:53.966678 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:32:53.966678 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:32:53.966678 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:32:53.966678 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:32:53.966678 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:32:53.966678 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:32:53.967779 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:32:53.967779 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:32:53.967779 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:32:53.967779 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:32:53.967779 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:32:53.967779 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:32:53.967779 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:32:53.967779 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:32:53.967779 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:32:53.967779 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:32:53.967779 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:32:53.967779 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:32:53.967779 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:32:53.967779 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:32:53.967779 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:32:53.967779 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:32:53.968614 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:32:53.968614 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:32:53.968614 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:32:53.968614 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:32:53.968614 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:32:53.968614 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:32:53.968614 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:32:53.968614 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:32:53.968614 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:32:53.968614 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:32:53.968614 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 13:32:53.968614 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:32:53.969213 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 129/253] 10.222.0.58 () {56 vars in 1096 bytes} [Tue Jul 26 13:32:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:32:53.970985 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9278bce-8191-4f80-8649-b505563164c7 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:53.971109 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9278bce-8191-4f80-8649-b505563164c7 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:53.971459 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9278bce-8191-4f80-8649-b505563164c7 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:53.977000 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 130/254] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:32:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:32:53.977000 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9278bce-8191-4f80-8649-b505563164c7 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:54.000283 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d3823c-83b0-45c2-8020-601c6d28a04a None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:54.049520 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f507abaf-ca57-448d-a8ac-a0db43b541c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:54.050440 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f507abaf-ca57-448d-a8ac-a0db43b541c9 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:54.050722 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f507abaf-ca57-448d-a8ac-a0db43b541c9 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:54.050997 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f507abaf-ca57-448d-a8ac-a0db43b541c9 None admin] PATH_INFO: `/v3/projects/6ad88492b683452a81ab7150c22b665e` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:54.052927 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f507abaf-ca57-448d-a8ac-a0db43b541c9 None admin] RBAC: Authorizing `identity:get_project(project_id=6ad88492b683452a81ab7150c22b665e)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:54.054447 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f507abaf-ca57-448d-a8ac-a0db43b541c9 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:54.061467 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 125/255] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Jul 26 13:32:53 2022] GET /identity/v3/projects/6ad88492b683452a81ab7150c22b665e => generated 302 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:54.061467 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f507abaf-ca57-448d-a8ac-a0db43b541c9 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:54.116258 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:54.122496 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9278bce-8191-4f80-8649-b505563164c7 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:54.162099 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0511a4-2fa5-4ae2-89b0-47e3bd55c597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:54.163185 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0511a4-2fa5-4ae2-89b0-47e3bd55c597 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:54.163703 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0511a4-2fa5-4ae2-89b0-47e3bd55c597 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:54.164222 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0511a4-2fa5-4ae2-89b0-47e3bd55c597 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:54.165257 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0511a4-2fa5-4ae2-89b0-47e3bd55c597 None admin] RBAC: Adding query filter params (role.id=39ccb3797eb542529c79507664d04b53, scope.project.id=6ad88492b683452a81ab7150c22b665e, user.id=1e91c3827fba42fe81d50be2d6525528) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:32:54.165784 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0511a4-2fa5-4ae2-89b0-47e3bd55c597 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:54.167604 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0511a4-2fa5-4ae2-89b0-47e3bd55c597 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:54.172982 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f507abaf-ca57-448d-a8ac-a0db43b541c9 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:54.174348 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 131/256] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Jul 26 13:32:53 2022] GET /identity/v3/role_assignments?user.id=1e91c3827fba42fe81d50be2d6525528&scope.project.id=6ad88492b683452a81ab7150c22b665e&role.id=39ccb3797eb542529c79507664d04b53 => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:54.176055 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53603f1c-88be-43fe-aa68-2e5c944fa4db None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:54.176540 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53603f1c-88be-43fe-aa68-2e5c944fa4db None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:54.177007 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53603f1c-88be-43fe-aa68-2e5c944fa4db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:54.198703 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-53603f1c-88be-43fe-aa68-2e5c944fa4db None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:32:54.200991 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-53603f1c-88be-43fe-aa68-2e5c944fa4db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:32:54.220100 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:54.247871 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72245d6b-dbd8-45ae-9146-e28adcca1b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:54.248843 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72245d6b-dbd8-45ae-9146-e28adcca1b31 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:54.249343 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72245d6b-dbd8-45ae-9146-e28adcca1b31 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:54.249885 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72245d6b-dbd8-45ae-9146-e28adcca1b31 None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:54.250723 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72245d6b-dbd8-45ae-9146-e28adcca1b31 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:32:54.251041 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72245d6b-dbd8-45ae-9146-e28adcca1b31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:54.252454 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72245d6b-dbd8-45ae-9146-e28adcca1b31 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:54.271612 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 126/257] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Jul 26 13:32:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:54.276670 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72245d6b-dbd8-45ae-9146-e28adcca1b31 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:54.287605 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 132/258] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:32:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 13:32:54.288827 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53603f1c-88be-43fe-aa68-2e5c944fa4db None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:54.411889 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:54.422915 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53603f1c-88be-43fe-aa68-2e5c944fa4db None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:54.463709 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bff595c-fa61-4969-91ef-d54a388a2db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:54.464753 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bff595c-fa61-4969-91ef-d54a388a2db2 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:54.464968 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72245d6b-dbd8-45ae-9146-e28adcca1b31 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:54.465428 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bff595c-fa61-4969-91ef-d54a388a2db2 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:54.465916 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bff595c-fa61-4969-91ef-d54a388a2db2 None admin] PATH_INFO: `/v3/users/b5a2469f69414f11896fc44a4f8536f1` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:54.467551 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bff595c-fa61-4969-91ef-d54a388a2db2 None admin] RBAC: Authorizing `identity:get_user(user_id=b5a2469f69414f11896fc44a4f8536f1)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:54.469265 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bff595c-fa61-4969-91ef-d54a388a2db2 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:54.471516 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 133/259] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Jul 26 13:32:54 2022] GET /identity/v3/users/b5a2469f69414f11896fc44a4f8536f1 => generated 283 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:54.472657 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bff595c-fa61-4969-91ef-d54a388a2db2 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:54.534413 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcd1522-a99c-48ce-828c-88aefa3882bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:54.535022 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcd1522-a99c-48ce-828c-88aefa3882bd None admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:54.535308 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcd1522-a99c-48ce-828c-88aefa3882bd None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:54.535589 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcd1522-a99c-48ce-828c-88aefa3882bd None admin] PATH_INFO: `/v3/projects/6ad88492b683452a81ab7150c22b665e/users/7ca36ada68fc48929486a18a402e7200/roles/759f389b221049c4b01faf3b984cd8d7` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:54.538334 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcd1522-a99c-48ce-828c-88aefa3882bd None admin] RBAC: Authorizing `identity:create_grant(project_id=6ad88492b683452a81ab7150c22b665e, user_id=7ca36ada68fc48929486a18a402e7200, role_id=759f389b221049c4b01faf3b984cd8d7)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:54.540823 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcd1522-a99c-48ce-828c-88aefa3882bd None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:54.576106 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 127/260] 10.222.0.58 () {58 vars in 1439 bytes} [Tue Jul 26 13:32:54 2022] PUT /identity/v3/projects/6ad88492b683452a81ab7150c22b665e/users/7ca36ada68fc48929486a18a402e7200/roles/759f389b221049c4b01faf3b984cd8d7 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:32:54.576841 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcd1522-a99c-48ce-828c-88aefa3882bd None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:54.601187 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bff595c-fa61-4969-91ef-d54a388a2db2 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:54.644495 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9a748c-8062-4159-9b80-bb40cea0f962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:54.645734 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9a748c-8062-4159-9b80-bb40cea0f962 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:54.646403 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9a748c-8062-4159-9b80-bb40cea0f962 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:54.646893 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9a748c-8062-4159-9b80-bb40cea0f962 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:54.649487 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:54.663464 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9a748c-8062-4159-9b80-bb40cea0f962 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:32:54.669905 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-2e9a748c-8062-4159-9b80-bb40cea0f962 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 13:32:54.669905 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:32:54.669905 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:32:54.669905 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:32:54.669905 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:32:54.669905 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:32:54.669905 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:32:54.669905 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:32:54.669905 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:32:54.669905 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:32:54.669905 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:32:54.669905 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:32:54.669905 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:32:54.669905 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:32:54.669905 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:32:54.669905 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:32:54.670841 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:32:54.670841 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:32:54.670841 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:32:54.670841 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:32:54.670841 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:32:54.670841 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:32:54.670841 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:32:54.670841 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:32:54.670841 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:32:54.670841 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:32:54.670841 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:32:54.670841 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:32:54.670841 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:32:54.670841 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:32:54.670841 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:32:54.670841 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:32:54.671634 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:32:54.671634 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:32:54.671634 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:32:54.671634 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:32:54.671634 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:32:54.671634 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:32:54.671634 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:32:54.671634 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:32:54.671634 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:32:54.671634 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:32:54.671634 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 13:32:54.671634 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:32:54.674531 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 134/261] 10.222.0.58 () {56 vars in 1114 bytes} [Tue Jul 26 13:32:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:32:54.681021 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9a748c-8062-4159-9b80-bb40cea0f962 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:54.713461 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:54.751090 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcd1522-a99c-48ce-828c-88aefa3882bd None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:54.805092 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9a748c-8062-4159-9b80-bb40cea0f962 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:54.835144 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea54b4e5-45ca-42e6-b741-72cce04dc304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:54.835843 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea54b4e5-45ca-42e6-b741-72cce04dc304 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:54.836132 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea54b4e5-45ca-42e6-b741-72cce04dc304 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:54.836412 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea54b4e5-45ca-42e6-b741-72cce04dc304 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:54.845912 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb5d9a3-d6a5-49b2-8ad3-053152fac382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:54.846870 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb5d9a3-d6a5-49b2-8ad3-053152fac382 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:54.847344 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb5d9a3-d6a5-49b2-8ad3-053152fac382 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:54.847810 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb5d9a3-d6a5-49b2-8ad3-053152fac382 None admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:54.848726 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb5d9a3-d6a5-49b2-8ad3-053152fac382 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:32:54.849362 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb5d9a3-d6a5-49b2-8ad3-053152fac382 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:54.850990 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb5d9a3-d6a5-49b2-8ad3-053152fac382 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:54.861845 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 135/262] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Jul 26 13:32:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:54.867866 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb5d9a3-d6a5-49b2-8ad3-053152fac382 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:54.949878 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea54b4e5-45ca-42e6-b741-72cce04dc304 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:54.951416 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea54b4e5-45ca-42e6-b741-72cce04dc304 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:54.963973 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:54.983640 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb5d9a3-d6a5-49b2-8ad3-053152fac382 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:55.019663 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:55.021543 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6869c63-42d7-4ae7-8db6-2d3417cb6157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:55.022418 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6869c63-42d7-4ae7-8db6-2d3417cb6157 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:55.022914 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6869c63-42d7-4ae7-8db6-2d3417cb6157 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:55.023425 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6869c63-42d7-4ae7-8db6-2d3417cb6157 None admin] PATH_INFO: `/v3/users/b5a2469f69414f11896fc44a4f8536f1` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:55.024821 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6869c63-42d7-4ae7-8db6-2d3417cb6157 None admin] RBAC: Authorizing `identity:get_user(user_id=b5a2469f69414f11896fc44a4f8536f1)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:55.026451 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6869c63-42d7-4ae7-8db6-2d3417cb6157 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:55.029257 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 136/263] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Jul 26 13:32:54 2022] GET /identity/v3/users/b5a2469f69414f11896fc44a4f8536f1 => generated 283 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:55.035117 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6869c63-42d7-4ae7-8db6-2d3417cb6157 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:55.138629 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 128/264] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Jul 26 13:32:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 562 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 13:32:55.144433 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea54b4e5-45ca-42e6-b741-72cce04dc304 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:55.147454 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6869c63-42d7-4ae7-8db6-2d3417cb6157 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:55.185183 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c200f3a-56ab-4902-b007-2f54a6d71d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:55.186196 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c200f3a-56ab-4902-b007-2f54a6d71d78 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:55.186695 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c200f3a-56ab-4902-b007-2f54a6d71d78 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:55.187264 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c200f3a-56ab-4902-b007-2f54a6d71d78 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:55.211632 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:55.262430 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c200f3a-56ab-4902-b007-2f54a6d71d78 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:55.264405 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c200f3a-56ab-4902-b007-2f54a6d71d78 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:55.346720 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea54b4e5-45ca-42e6-b741-72cce04dc304 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:55.389625 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 137/265] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Jul 26 13:32:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 26 13:32:55.400749 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c200f3a-56ab-4902-b007-2f54a6d71d78 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:55.474246 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc75542-61bc-4eec-bbf8-2db161786a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:55.474930 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc75542-61bc-4eec-bbf8-2db161786a36 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:55.475218 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc75542-61bc-4eec-bbf8-2db161786a36 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:55.475500 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc75542-61bc-4eec-bbf8-2db161786a36 None admin] PATH_INFO: `/v3/projects/6ad88492b683452a81ab7150c22b665e` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:55.477882 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:55.482396 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc75542-61bc-4eec-bbf8-2db161786a36 None admin] RBAC: Authorizing `identity:get_project(project_id=6ad88492b683452a81ab7150c22b665e)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:55.484328 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc75542-61bc-4eec-bbf8-2db161786a36 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:55.486078 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 129/266] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Jul 26 13:32:55 2022] GET /identity/v3/projects/6ad88492b683452a81ab7150c22b665e => generated 302 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:32:55.492789 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc75542-61bc-4eec-bbf8-2db161786a36 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:55.547695 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:55.585186 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c200f3a-56ab-4902-b007-2f54a6d71d78 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:55.662058 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289d9dd0-2140-4103-91c9-2843dd326611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:55.663131 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d9dd0-2140-4103-91c9-2843dd326611 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:55.663620 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d9dd0-2140-4103-91c9-2843dd326611 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:55.664094 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d9dd0-2140-4103-91c9-2843dd326611 None admin] PATH_INFO: `/v3/projects/6ad88492b683452a81ab7150c22b665e` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:55.665696 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289d9dd0-2140-4103-91c9-2843dd326611 None admin] RBAC: Authorizing `identity:get_project(project_id=6ad88492b683452a81ab7150c22b665e)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:55.667491 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289d9dd0-2140-4103-91c9-2843dd326611 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:55.673832 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 138/267] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Jul 26 13:32:55 2022] GET /identity/v3/projects/6ad88492b683452a81ab7150c22b665e => generated 302 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:32:55.684507 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289d9dd0-2140-4103-91c9-2843dd326611 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:55.719526 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc75542-61bc-4eec-bbf8-2db161786a36 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:55.800097 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589f83aa-7421-451e-b642-4d75d28d6766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:55.804155 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589f83aa-7421-451e-b642-4d75d28d6766 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:55.804683 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589f83aa-7421-451e-b642-4d75d28d6766 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:55.805156 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589f83aa-7421-451e-b642-4d75d28d6766 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:55.806202 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589f83aa-7421-451e-b642-4d75d28d6766 None admin] RBAC: Adding query filter params (role.id=6e0201a9a6484f7d8b112b1eafff70bd, scope.project.id=6ad88492b683452a81ab7150c22b665e, user.id=b5a2469f69414f11896fc44a4f8536f1) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:32:55.806945 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589f83aa-7421-451e-b642-4d75d28d6766 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:55.810274 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589f83aa-7421-451e-b642-4d75d28d6766 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:55.824216 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 130/268] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Jul 26 13:32:55 2022] GET /identity/v3/role_assignments?user.id=b5a2469f69414f11896fc44a4f8536f1&scope.project.id=6ad88492b683452a81ab7150c22b665e&role.id=6e0201a9a6484f7d8b112b1eafff70bd => generated 608 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:32:55.892888 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289d9dd0-2140-4103-91c9-2843dd326611 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:55.926783 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:55.946484 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75955a0f-2207-4dda-9ff8-c11e91a17124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:55.947538 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75955a0f-2207-4dda-9ff8-c11e91a17124 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:55.948030 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75955a0f-2207-4dda-9ff8-c11e91a17124 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:55.948517 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75955a0f-2207-4dda-9ff8-c11e91a17124 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:55.949639 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75955a0f-2207-4dda-9ff8-c11e91a17124 None admin] RBAC: Adding query filter params (role.id=5cb730ec786743c3bb8af83da90f9309, scope.project.id=6ad88492b683452a81ab7150c22b665e, user.id=b5a2469f69414f11896fc44a4f8536f1) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:32:55.950183 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75955a0f-2207-4dda-9ff8-c11e91a17124 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:55.951953 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75955a0f-2207-4dda-9ff8-c11e91a17124 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:55.962941 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 139/269] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Jul 26 13:32:55 2022] GET /identity/v3/role_assignments?user.id=b5a2469f69414f11896fc44a4f8536f1&scope.project.id=6ad88492b683452a81ab7150c22b665e&role.id=5cb730ec786743c3bb8af83da90f9309 => generated 608 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:55.965936 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35237c2-8b27-410c-8ec1-0006991bd71b None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:55.966505 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35237c2-8b27-410c-8ec1-0006991bd71b None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:55.966982 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35237c2-8b27-410c-8ec1-0006991bd71b None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:55.968697 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 131/270] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:32:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:32:55.975036 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068b539c-43d2-48e5-bc6b-bd7bff97bf51 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:55.975665 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068b539c-43d2-48e5-bc6b-bd7bff97bf51 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:55.976142 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068b539c-43d2-48e5-bc6b-bd7bff97bf51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:55.997537 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-068b539c-43d2-48e5-bc6b-bd7bff97bf51 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:32:55.999343 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-068b539c-43d2-48e5-bc6b-bd7bff97bf51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:32:56.104731 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 140/271] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:32:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 13:32:56.116997 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdc15b9-3c7e-4d0d-8778-12b660285bff None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:56.118094 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdc15b9-3c7e-4d0d-8778-12b660285bff None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:56.118583 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdc15b9-3c7e-4d0d-8778-12b660285bff None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:56.120781 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 132/272] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:32:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:32:56.127526 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad7432d-ae58-434d-8fa1-e6891483c93a None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:56.128437 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad7432d-ae58-434d-8fa1-e6891483c93a None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:56.129204 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad7432d-ae58-434d-8fa1-e6891483c93a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:56.158791 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-aad7432d-ae58-434d-8fa1-e6891483c93a None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:32:56.161145 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-aad7432d-ae58-434d-8fa1-e6891483c93a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:32:56.170251 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdc15b9-3c7e-4d0d-8778-12b660285bff None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:56.259892 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 141/273] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:32:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 13:32:56.286137 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad7432d-ae58-434d-8fa1-e6891483c93a None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:56.300760 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:56.318656 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdc15b9-3c7e-4d0d-8778-12b660285bff None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:56.361124 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7fd87f-b1ff-4465-9816-eb26865afaf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:56.361820 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7fd87f-b1ff-4465-9816-eb26865afaf8 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:56.362106 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7fd87f-b1ff-4465-9816-eb26865afaf8 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:56.362384 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7fd87f-b1ff-4465-9816-eb26865afaf8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:56.362885 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:56.363813 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7fd87f-b1ff-4465-9816-eb26865afaf8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:56.365340 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7fd87f-b1ff-4465-9816-eb26865afaf8 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:56.368112 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 133/274] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:32:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:32:56.374665 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7fd87f-b1ff-4465-9816-eb26865afaf8 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:56.452009 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad7432d-ae58-434d-8fa1-e6891483c93a None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:56.517058 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e60d42-3eab-4152-a3f6-4dd8f84e7445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:56.517684 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e60d42-3eab-4152-a3f6-4dd8f84e7445 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:56.517965 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e60d42-3eab-4152-a3f6-4dd8f84e7445 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:56.518239 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e60d42-3eab-4152-a3f6-4dd8f84e7445 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:56.522836 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7fd87f-b1ff-4465-9816-eb26865afaf8 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:56.533934 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e60d42-3eab-4152-a3f6-4dd8f84e7445 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:32:56.540313 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-11e60d42-3eab-4152-a3f6-4dd8f84e7445 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 13:32:56.540313 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:32:56.540313 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:32:56.540313 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:32:56.540313 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:32:56.540313 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:32:56.540313 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:32:56.540313 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:32:56.540313 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:32:56.540313 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:32:56.540313 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:32:56.540313 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:32:56.540313 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:32:56.540313 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:32:56.540313 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:32:56.540313 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:32:56.541238 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:32:56.541238 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:32:56.541238 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:32:56.541238 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:32:56.541238 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:32:56.541238 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:32:56.541238 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:32:56.541238 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:32:56.541238 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:32:56.541238 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:32:56.541238 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:32:56.541238 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:32:56.541238 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:32:56.541238 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:32:56.541238 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:32:56.541238 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:32:56.542055 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:32:56.542055 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:32:56.542055 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:32:56.542055 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:32:56.542055 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:32:56.542055 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:32:56.542055 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:32:56.542055 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:32:56.542055 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:32:56.542055 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:32:56.542055 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 13:32:56.542055 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:32:56.542666 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 142/275] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Jul 26 13:32:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 13:32:56.548454 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e60d42-3eab-4152-a3f6-4dd8f84e7445 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:56.567705 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d53d8ba-3b96-4b1f-a92f-1341ef1271c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:56.568446 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d53d8ba-3b96-4b1f-a92f-1341ef1271c8 None admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:56.568818 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d53d8ba-3b96-4b1f-a92f-1341ef1271c8 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:56.569166 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d53d8ba-3b96-4b1f-a92f-1341ef1271c8 None admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:56.569978 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d53d8ba-3b96-4b1f-a92f-1341ef1271c8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:56.571563 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d53d8ba-3b96-4b1f-a92f-1341ef1271c8 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:56.597451 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 134/276] 10.222.0.58 () {60 vars in 1132 bytes} [Tue Jul 26 13:32:56 2022] POST /identity/v3/users => generated 293 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:32:56.681041 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e60d42-3eab-4152-a3f6-4dd8f84e7445 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:56.684093 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:56.723128 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c699e4b5-afaf-423a-875e-29d26f5bec93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:56.723777 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c699e4b5-afaf-423a-875e-29d26f5bec93 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:56.724056 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c699e4b5-afaf-423a-875e-29d26f5bec93 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:56.724332 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c699e4b5-afaf-423a-875e-29d26f5bec93 None admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:56.725128 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c699e4b5-afaf-423a-875e-29d26f5bec93 None admin] RBAC: Adding query filter params (name=member) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:32:56.725434 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c699e4b5-afaf-423a-875e-29d26f5bec93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:56.726980 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c699e4b5-afaf-423a-875e-29d26f5bec93 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:56.736799 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 143/277] 10.222.0.58 () {56 vars in 1101 bytes} [Tue Jul 26 13:32:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:32:56.742663 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d53d8ba-3b96-4b1f-a92f-1341ef1271c8 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:56.757875 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:56.867979 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d53d8ba-3b96-4b1f-a92f-1341ef1271c8 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:56.920414 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecd4e47-c727-48c6-ba10-fb8925b7318a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:56.921088 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecd4e47-c727-48c6-ba10-fb8925b7318a None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:56.921379 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecd4e47-c727-48c6-ba10-fb8925b7318a None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:56.921694 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecd4e47-c727-48c6-ba10-fb8925b7318a None admin] PATH_INFO: `/v3/users/362c0984745e40f887648a4fe6ef2ea8` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:56.939298 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecd4e47-c727-48c6-ba10-fb8925b7318a None admin] RBAC: Authorizing `identity:get_user(user_id=362c0984745e40f887648a4fe6ef2ea8)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:56.940776 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecd4e47-c727-48c6-ba10-fb8925b7318a None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:56.942637 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 135/278] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Jul 26 13:32:56 2022] GET /identity/v3/users/362c0984745e40f887648a4fe6ef2ea8 => generated 293 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:56.949962 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c699e4b5-afaf-423a-875e-29d26f5bec93 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:57.076432 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c699e4b5-afaf-423a-875e-29d26f5bec93 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:57.122125 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251dca66-2fe3-455d-8012-8fd251dc30c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:57.122688 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251dca66-2fe3-455d-8012-8fd251dc30c3 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:57.122974 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251dca66-2fe3-455d-8012-8fd251dc30c3 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:57.123256 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251dca66-2fe3-455d-8012-8fd251dc30c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:57.124068 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251dca66-2fe3-455d-8012-8fd251dc30c3 None admin] RBAC: Adding query filter params (role.id=759f389b221049c4b01faf3b984cd8d7, user.id=362c0984745e40f887648a4fe6ef2ea8, scope.system=all) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:32:57.124403 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251dca66-2fe3-455d-8012-8fd251dc30c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:57.126001 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251dca66-2fe3-455d-8012-8fd251dc30c3 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:57.132136 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 144/279] 10.222.0.58 () {56 vars in 1308 bytes} [Tue Jul 26 13:32:56 2022] GET /identity/v3/role_assignments?user.id=362c0984745e40f887648a4fe6ef2ea8&scope.system=all&role.id=759f389b221049c4b01faf3b984cd8d7 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:57.133302 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508076da-1ede-4b10-ac08-4af53df7ee45 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:57.133789 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508076da-1ede-4b10-ac08-4af53df7ee45 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:57.134017 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508076da-1ede-4b10-ac08-4af53df7ee45 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:57.135496 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 136/280] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:32:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:32:57.143720 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7fd54a-d1e8-4c69-a8d4-792643c289e7 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:57.144047 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7fd54a-d1e8-4c69-a8d4-792643c289e7 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:57.144287 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7fd54a-d1e8-4c69-a8d4-792643c289e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:57.167195 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-ff7fd54a-d1e8-4c69-a8d4-792643c289e7 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:32:57.168470 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-ff7fd54a-d1e8-4c69-a8d4-792643c289e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:32:57.197839 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:57.262204 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 145/281] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:32:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 13:32:57.281092 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508076da-1ede-4b10-ac08-4af53df7ee45 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:57.386193 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:57.474272 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508076da-1ede-4b10-ac08-4af53df7ee45 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:57.524673 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2e49a5-ef8e-4457-8a91-3f2ae0c5d7f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:57.525326 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2e49a5-ef8e-4457-8a91-3f2ae0c5d7f4 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:57.525633 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2e49a5-ef8e-4457-8a91-3f2ae0c5d7f4 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:57.525907 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2e49a5-ef8e-4457-8a91-3f2ae0c5d7f4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:57.542436 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2e49a5-ef8e-4457-8a91-3f2ae0c5d7f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:32:57.547968 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-4a2e49a5-ef8e-4457-8a91-3f2ae0c5d7f4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 13:32:57.547968 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:32:57.547968 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:32:57.547968 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:32:57.547968 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:32:57.547968 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:32:57.547968 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:32:57.547968 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:32:57.547968 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:32:57.547968 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:32:57.547968 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:32:57.547968 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:32:57.547968 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:32:57.547968 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:32:57.547968 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:32:57.547968 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:32:57.548832 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:32:57.548832 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:32:57.548832 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:32:57.548832 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:32:57.548832 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:32:57.548832 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:32:57.548832 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:32:57.548832 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:32:57.548832 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:32:57.548832 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:32:57.548832 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:32:57.548832 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:32:57.548832 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:32:57.548832 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:32:57.548832 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:32:57.548832 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:32:57.549640 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:32:57.549640 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:32:57.549640 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:32:57.549640 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:32:57.549640 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:32:57.549640 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:32:57.549640 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:32:57.549640 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:32:57.549640 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:32:57.549640 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:32:57.549640 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 13:32:57.549640 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:32:57.553613 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 137/282] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Jul 26 13:32:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:32:57.555813 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7fd54a-d1e8-4c69-a8d4-792643c289e7 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:57.598145 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054642e3-4ece-4be6-8cab-dde4b98ae11b None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:57.598487 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054642e3-4ece-4be6-8cab-dde4b98ae11b None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:57.598689 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054642e3-4ece-4be6-8cab-dde4b98ae11b None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:57.600364 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 138/283] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:32:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:32:57.607294 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c564ab52-91e9-451c-8a99-752c69a81443 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:57.607574 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c564ab52-91e9-451c-8a99-752c69a81443 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:57.607836 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c564ab52-91e9-451c-8a99-752c69a81443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:57.631120 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-c564ab52-91e9-451c-8a99-752c69a81443 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:32:57.632976 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-c564ab52-91e9-451c-8a99-752c69a81443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:32:57.744557 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7fd54a-d1e8-4c69-a8d4-792643c289e7 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:57.775206 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 139/284] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:32:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 13:32:57.781757 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe4dedd-a20d-4959-a703-e527c335c53e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:57.782401 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe4dedd-a20d-4959-a703-e527c335c53e None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:57.782687 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe4dedd-a20d-4959-a703-e527c335c53e None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:57.782964 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe4dedd-a20d-4959-a703-e527c335c53e None admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:57.783694 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe4dedd-a20d-4959-a703-e527c335c53e None admin] RBAC: Adding query filter params (name=reader) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:32:57.784001 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe4dedd-a20d-4959-a703-e527c335c53e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:57.785372 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe4dedd-a20d-4959-a703-e527c335c53e None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:57.800170 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 146/285] 10.222.0.58 () {56 vars in 1101 bytes} [Tue Jul 26 13:32:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:57.801665 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c564ab52-91e9-451c-8a99-752c69a81443 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:57.810998 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe4dedd-a20d-4959-a703-e527c335c53e None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:57.858384 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:57.893263 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:57.988979 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c564ab52-91e9-451c-8a99-752c69a81443 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:58.026849 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe4dedd-a20d-4959-a703-e527c335c53e None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:58.068282 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70ae8df-3631-40e9-827d-8287eddbd7af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:58.068948 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70ae8df-3631-40e9-827d-8287eddbd7af None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:58.069229 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70ae8df-3631-40e9-827d-8287eddbd7af None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:58.069531 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70ae8df-3631-40e9-827d-8287eddbd7af None admin] PATH_INFO: `/v3/users/1e91c3827fba42fe81d50be2d6525528` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:58.070872 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70ae8df-3631-40e9-827d-8287eddbd7af None admin] RBAC: Authorizing `identity:get_user(user_id=1e91c3827fba42fe81d50be2d6525528)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:58.072325 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70ae8df-3631-40e9-827d-8287eddbd7af None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:58.076953 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 140/286] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Jul 26 13:32:57 2022] GET /identity/v3/users/1e91c3827fba42fe81d50be2d6525528 => generated 297 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:58.081274 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70ae8df-3631-40e9-827d-8287eddbd7af None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:58.103331 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeabb80-25fa-4119-b7eb-95bc57c1ac48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:58.104000 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeabb80-25fa-4119-b7eb-95bc57c1ac48 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:58.104332 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeabb80-25fa-4119-b7eb-95bc57c1ac48 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:58.104611 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeabb80-25fa-4119-b7eb-95bc57c1ac48 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:58.127980 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeabb80-25fa-4119-b7eb-95bc57c1ac48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:32:58.137105 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-baeabb80-25fa-4119-b7eb-95bc57c1ac48 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 13:32:58.137105 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:32:58.137105 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:32:58.137105 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:32:58.137105 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:32:58.137105 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:32:58.137105 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:32:58.137105 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:32:58.137105 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:32:58.137105 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:32:58.137105 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:32:58.137105 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:32:58.137105 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:32:58.137105 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:32:58.137105 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:32:58.137105 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:32:58.138102 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:32:58.138102 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:32:58.138102 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:32:58.138102 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:32:58.138102 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:32:58.138102 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:32:58.138102 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:32:58.138102 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:32:58.138102 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:32:58.138102 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:32:58.138102 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:32:58.138102 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:32:58.138102 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:32:58.138102 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:32:58.138102 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:32:58.138102 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:32:58.139333 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:32:58.139333 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:32:58.139333 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:32:58.139333 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:32:58.139333 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:32:58.139333 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:32:58.139333 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:32:58.139333 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:32:58.139333 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:32:58.139333 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:32:58.139333 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 13:32:58.139333 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:32:58.139333 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 147/287] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Jul 26 13:32:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 13:32:58.144789 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeabb80-25fa-4119-b7eb-95bc57c1ac48 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:58.254506 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:58.293328 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70ae8df-3631-40e9-827d-8287eddbd7af None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:58.305557 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeabb80-25fa-4119-b7eb-95bc57c1ac48 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:58.316144 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:58.332386 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fcda19-d51d-4361-ae9e-0d52a2a95d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:58.333322 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fcda19-d51d-4361-ae9e-0d52a2a95d03 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:58.333776 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fcda19-d51d-4361-ae9e-0d52a2a95d03 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:58.334202 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fcda19-d51d-4361-ae9e-0d52a2a95d03 None admin] PATH_INFO: `/v3/projects/6ad88492b683452a81ab7150c22b665e` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:58.335533 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fcda19-d51d-4361-ae9e-0d52a2a95d03 None admin] RBAC: Authorizing `identity:get_project(project_id=6ad88492b683452a81ab7150c22b665e)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:58.337139 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fcda19-d51d-4361-ae9e-0d52a2a95d03 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:58.339387 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 141/288] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Jul 26 13:32:58 2022] GET /identity/v3/projects/6ad88492b683452a81ab7150c22b665e => generated 302 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:58.345311 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fcda19-d51d-4361-ae9e-0d52a2a95d03 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:58.402902 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090eb7a7-95ce-4579-a7d1-8f694e82b0ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:58.403555 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090eb7a7-95ce-4579-a7d1-8f694e82b0ce None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:58.403842 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090eb7a7-95ce-4579-a7d1-8f694e82b0ce None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:58.404128 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090eb7a7-95ce-4579-a7d1-8f694e82b0ce None admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:58.404894 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090eb7a7-95ce-4579-a7d1-8f694e82b0ce None admin] RBAC: Adding query filter params (name=member) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:32:58.405205 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090eb7a7-95ce-4579-a7d1-8f694e82b0ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:58.406635 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090eb7a7-95ce-4579-a7d1-8f694e82b0ce None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:58.425454 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 148/289] 10.222.0.58 () {56 vars in 1101 bytes} [Tue Jul 26 13:32:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:32:58.432014 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090eb7a7-95ce-4579-a7d1-8f694e82b0ce None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:58.486845 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fcda19-d51d-4361-ae9e-0d52a2a95d03 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:58.576563 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978480e6-e932-46ce-8428-8084d8719a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:58.577501 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978480e6-e932-46ce-8428-8084d8719a8f None admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:58.577930 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978480e6-e932-46ce-8428-8084d8719a8f None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:58.578372 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978480e6-e932-46ce-8428-8084d8719a8f None admin] PATH_INFO: `/v3/projects/6ad88492b683452a81ab7150c22b665e/users/1e91c3827fba42fe81d50be2d6525528/roles/39ccb3797eb542529c79507664d04b53` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:58.585399 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978480e6-e932-46ce-8428-8084d8719a8f None admin] RBAC: Authorizing `identity:create_grant(project_id=6ad88492b683452a81ab7150c22b665e, user_id=1e91c3827fba42fe81d50be2d6525528, role_id=39ccb3797eb542529c79507664d04b53)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:58.587308 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978480e6-e932-46ce-8428-8084d8719a8f None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:58.612730 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 142/290] 10.222.0.58 () {58 vars in 1439 bytes} [Tue Jul 26 13:32:58 2022] PUT /identity/v3/projects/6ad88492b683452a81ab7150c22b665e/users/1e91c3827fba42fe81d50be2d6525528/roles/39ccb3797eb542529c79507664d04b53 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:32:58.614900 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090eb7a7-95ce-4579-a7d1-8f694e82b0ce None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:58.674667 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c77883-e0d1-400a-a15a-720910cfc9d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:58.675828 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c77883-e0d1-400a-a15a-720910cfc9d0 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:58.676255 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c77883-e0d1-400a-a15a-720910cfc9d0 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:58.676720 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c77883-e0d1-400a-a15a-720910cfc9d0 None admin] PATH_INFO: `/v3/users/7ca36ada68fc48929486a18a402e7200` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:58.681469 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c77883-e0d1-400a-a15a-720910cfc9d0 None admin] RBAC: Authorizing `identity:get_user(user_id=7ca36ada68fc48929486a18a402e7200)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:58.683186 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c77883-e0d1-400a-a15a-720910cfc9d0 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:58.686313 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 149/291] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Jul 26 13:32:58 2022] GET /identity/v3/users/7ca36ada68fc48929486a18a402e7200 => generated 297 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:32:58.692253 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978480e6-e932-46ce-8428-8084d8719a8f None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:58.768022 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:58.828908 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978480e6-e932-46ce-8428-8084d8719a8f None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:58.871743 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c456d78a-084f-45ac-a0ee-fdd0d3ec2e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:58.872631 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c456d78a-084f-45ac-a0ee-fdd0d3ec2e07 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:58.873061 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c456d78a-084f-45ac-a0ee-fdd0d3ec2e07 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:58.873505 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c456d78a-084f-45ac-a0ee-fdd0d3ec2e07 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:58.951531 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c456d78a-084f-45ac-a0ee-fdd0d3ec2e07 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:58.953295 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c456d78a-084f-45ac-a0ee-fdd0d3ec2e07 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:59.006860 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:59.064343 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 143/292] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Jul 26 13:32:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 13:32:59.070006 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c77883-e0d1-400a-a15a-720910cfc9d0 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:59.086400 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:59.203746 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c77883-e0d1-400a-a15a-720910cfc9d0 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:59.249829 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0d8758-fb6e-4a92-bda9-8f1a62d03076 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:59.250115 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0d8758-fb6e-4a92-bda9-8f1a62d03076 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:59.250395 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0d8758-fb6e-4a92-bda9-8f1a62d03076 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:59.251979 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 144/293] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:32:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 13:32:59.259269 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c84c96d-8304-4447-8f0c-78ea393309c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:59.259924 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c84c96d-8304-4447-8f0c-78ea393309c3 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:59.260214 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c84c96d-8304-4447-8f0c-78ea393309c3 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:59.262497 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a23c3a-9585-4b0a-ad52-8afe58a22336 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:59.262839 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a23c3a-9585-4b0a-ad52-8afe58a22336 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:59.263110 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a23c3a-9585-4b0a-ad52-8afe58a22336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:59.268448 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c84c96d-8304-4447-8f0c-78ea393309c3 None admin] PATH_INFO: `/v3/projects/6ad88492b683452a81ab7150c22b665e` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:59.277772 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c84c96d-8304-4447-8f0c-78ea393309c3 None admin] RBAC: Authorizing `identity:get_project(project_id=6ad88492b683452a81ab7150c22b665e)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:59.279247 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c84c96d-8304-4447-8f0c-78ea393309c3 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:59.285245 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 150/294] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Jul 26 13:32:59 2022] GET /identity/v3/projects/6ad88492b683452a81ab7150c22b665e => generated 302 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:59.287928 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c84c96d-8304-4447-8f0c-78ea393309c3 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:59.294503 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-58a23c3a-9585-4b0a-ad52-8afe58a22336 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:32:59.295732 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-58a23c3a-9585-4b0a-ad52-8afe58a22336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:32:59.434787 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 145/295] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:32:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 13:32:59.445049 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:59.470005 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c84c96d-8304-4447-8f0c-78ea393309c3 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:59.474923 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a23c3a-9585-4b0a-ad52-8afe58a22336 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:59.541835 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db87558d-8a71-4b7e-9ab3-f1cec9debe49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:59.543399 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db87558d-8a71-4b7e-9ab3-f1cec9debe49 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:59.543921 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db87558d-8a71-4b7e-9ab3-f1cec9debe49 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:59.544408 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db87558d-8a71-4b7e-9ab3-f1cec9debe49 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:59.545413 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db87558d-8a71-4b7e-9ab3-f1cec9debe49 None admin] RBAC: Adding query filter params (role.id=759f389b221049c4b01faf3b984cd8d7, scope.project.id=6ad88492b683452a81ab7150c22b665e, user.id=7ca36ada68fc48929486a18a402e7200) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:32:59.545954 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db87558d-8a71-4b7e-9ab3-f1cec9debe49 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:59.547594 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db87558d-8a71-4b7e-9ab3-f1cec9debe49 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:59.563021 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 151/296] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Jul 26 13:32:59 2022] GET /identity/v3/role_assignments?user.id=7ca36ada68fc48929486a18a402e7200&scope.project.id=6ad88492b683452a81ab7150c22b665e&role.id=759f389b221049c4b01faf3b984cd8d7 => generated 608 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:59.574375 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b24a023-b6b9-426e-954e-4561342b40f4 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:59.574994 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b24a023-b6b9-426e-954e-4561342b40f4 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:59.575465 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b24a023-b6b9-426e-954e-4561342b40f4 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:59.577296 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 152/297] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:32:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:32:59.582994 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c089bf-b7f7-44fc-a41c-497c29644aa4 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:59.583598 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c089bf-b7f7-44fc-a41c-497c29644aa4 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:59.584073 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c089bf-b7f7-44fc-a41c-497c29644aa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:59.600645 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:59.609815 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-a6c089bf-b7f7-44fc-a41c-497c29644aa4 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:32:59.611376 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-a6c089bf-b7f7-44fc-a41c-497c29644aa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:32:59.632634 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a23c3a-9585-4b0a-ad52-8afe58a22336 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:59.675890 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee44d0b-1457-4fe1-9621-2ece7d268f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:59.676872 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee44d0b-1457-4fe1-9621-2ece7d268f78 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:59.677291 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee44d0b-1457-4fe1-9621-2ece7d268f78 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:59.677746 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee44d0b-1457-4fe1-9621-2ece7d268f78 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:59.691719 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee44d0b-1457-4fe1-9621-2ece7d268f78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:32:59.697997 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-9ee44d0b-1457-4fe1-9621-2ece7d268f78 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 13:32:59.697997 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:32:59.697997 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:32:59.697997 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:32:59.697997 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:32:59.697997 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:32:59.697997 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:32:59.697997 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:32:59.697997 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:32:59.697997 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:32:59.697997 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:32:59.697997 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:32:59.697997 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:32:59.697997 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:32:59.697997 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:32:59.697997 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:32:59.699782 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:32:59.699782 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:32:59.699782 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:32:59.699782 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:32:59.699782 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:32:59.699782 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:32:59.699782 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:32:59.699782 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:32:59.699782 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:32:59.699782 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:32:59.699782 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:32:59.699782 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:32:59.699782 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:32:59.699782 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:32:59.699782 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:32:59.699782 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:32:59.701263 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:32:59.701263 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:32:59.701263 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:32:59.701263 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:32:59.701263 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:32:59.701263 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:32:59.701263 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:32:59.701263 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:32:59.701263 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:32:59.701263 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:32:59.701263 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 13:32:59.701263 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:32:59.704127 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 146/298] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Jul 26 13:32:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 13:32:59.710506 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc8454c-d1c5-41a4-87ef-0e85766be04d None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:59.711023 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc8454c-d1c5-41a4-87ef-0e85766be04d None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:59.713179 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc8454c-d1c5-41a4-87ef-0e85766be04d None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:59.713179 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 147/299] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:32:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:32:59.714410 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc8454c-d1c5-41a4-87ef-0e85766be04d None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:59.745518 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 153/300] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:32:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 13:32:59.747574 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b128c71f-3ded-4f8d-aa93-78d6057ee675 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:59.754069 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b128c71f-3ded-4f8d-aa93-78d6057ee675 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:59.754470 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b128c71f-3ded-4f8d-aa93-78d6057ee675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:59.786524 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-b128c71f-3ded-4f8d-aa93-78d6057ee675 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:32:59.787144 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-b128c71f-3ded-4f8d-aa93-78d6057ee675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:32:59.852664 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc8454c-d1c5-41a4-87ef-0e85766be04d None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:32:59.881104 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 154/301] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:32:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 13:32:59.882366 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b128c71f-3ded-4f8d-aa93-78d6057ee675 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:32:59.887028 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:59.904456 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfb5cbe-8206-49b3-9aa5-471863de5b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:32:59.905078 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfb5cbe-8206-49b3-9aa5-471863de5b63 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:32:59.905363 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfb5cbe-8206-49b3-9aa5-471863de5b63 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:32:59.905672 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfb5cbe-8206-49b3-9aa5-471863de5b63 None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:32:59.906391 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfb5cbe-8206-49b3-9aa5-471863de5b63 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:32:59.906694 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfb5cbe-8206-49b3-9aa5-471863de5b63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:32:59.908079 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfb5cbe-8206-49b3-9aa5-471863de5b63 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:32:59.913682 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:32:59.918136 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 148/302] 10.222.0.58 () {56 vars in 1101 bytes} [Tue Jul 26 13:32:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:32:59.919293 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfb5cbe-8206-49b3-9aa5-471863de5b63 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:00.011732 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b128c71f-3ded-4f8d-aa93-78d6057ee675 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:00.041904 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfb5cbe-8206-49b3-9aa5-471863de5b63 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:00.053892 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f6de56-e7f2-4f14-bc31-a39e0cb0becb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:00.054549 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f6de56-e7f2-4f14-bc31-a39e0cb0becb None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:00.054836 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f6de56-e7f2-4f14-bc31-a39e0cb0becb None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:00.055111 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f6de56-e7f2-4f14-bc31-a39e0cb0becb None admin] PATH_INFO: `/v3/roles/member` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:00.067347 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f6de56-e7f2-4f14-bc31-a39e0cb0becb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:33:00.072537 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-a3f6de56-e7f2-4f14-bc31-a39e0cb0becb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 13:33:00.072537 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:00.072537 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:00.072537 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:00.072537 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:00.072537 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:00.072537 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:00.072537 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:00.072537 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:00.072537 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:00.072537 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:00.072537 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:00.072537 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:33:00.072537 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:33:00.072537 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:33:00.072537 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:33:00.073979 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:33:00.073979 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:33:00.073979 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:33:00.073979 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:33:00.073979 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:00.073979 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:00.073979 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:00.073979 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:00.073979 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:00.073979 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:00.073979 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:00.073979 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:00.073979 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:00.073979 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:00.073979 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:00.073979 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:00.075285 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:00.075285 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:00.075285 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:00.075285 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:00.075285 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:33:00.075285 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:33:00.075285 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:33:00.075285 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:33:00.075285 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:33:00.075285 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:33:00.075285 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 13:33:00.075285 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:33:00.075285 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 155/303] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Jul 26 13:32:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:00.075285 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f6de56-e7f2-4f14-bc31-a39e0cb0becb None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:00.086636 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7f6008-bfbe-4ad7-becc-621085c1791f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:00.087290 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7f6008-bfbe-4ad7-becc-621085c1791f None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:00.087573 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7f6008-bfbe-4ad7-becc-621085c1791f None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:00.087851 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7f6008-bfbe-4ad7-becc-621085c1791f None admin] PATH_INFO: `/v3/domains/default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:00.089006 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7f6008-bfbe-4ad7-becc-621085c1791f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:00.091028 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7f6008-bfbe-4ad7-becc-621085c1791f None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:00.092960 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 149/304] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:32:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:00.093683 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7f6008-bfbe-4ad7-becc-621085c1791f None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:00.152899 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:00.168799 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:00.204824 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f6de56-e7f2-4f14-bc31-a39e0cb0becb None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:00.209098 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7f6008-bfbe-4ad7-becc-621085c1791f None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:00.250748 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f828c766-79c1-4904-9fca-3694486a042d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:00.251427 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f828c766-79c1-4904-9fca-3694486a042d None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:00.251709 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f828c766-79c1-4904-9fca-3694486a042d None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:00.251984 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f828c766-79c1-4904-9fca-3694486a042d None admin] PATH_INFO: `/v3/users/a789c6c8fc6f48ac9a5b9c1295b5fea1` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:00.252558 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920b73d9-0a3e-4e09-a855-b88911cc9204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:00.253181 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920b73d9-0a3e-4e09-a855-b88911cc9204 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:00.253463 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920b73d9-0a3e-4e09-a855-b88911cc9204 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:00.253823 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920b73d9-0a3e-4e09-a855-b88911cc9204 None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:00.254547 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920b73d9-0a3e-4e09-a855-b88911cc9204 None admin] RBAC: Adding query filter params (name=member) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:00.254852 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920b73d9-0a3e-4e09-a855-b88911cc9204 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:00.256236 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920b73d9-0a3e-4e09-a855-b88911cc9204 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:00.265168 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 150/305] 10.222.0.58 () {56 vars in 1101 bytes} [Tue Jul 26 13:33:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:00.266016 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920b73d9-0a3e-4e09-a855-b88911cc9204 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:00.273050 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f828c766-79c1-4904-9fca-3694486a042d None admin] RBAC: Authorizing `identity:get_user(user_id=a789c6c8fc6f48ac9a5b9c1295b5fea1)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:00.274576 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f828c766-79c1-4904-9fca-3694486a042d None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:00.276453 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 156/306] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Jul 26 13:33:00 2022] GET /identity/v3/users/a789c6c8fc6f48ac9a5b9c1295b5fea1 => generated 293 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:00.277656 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f828c766-79c1-4904-9fca-3694486a042d None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:00.391854 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920b73d9-0a3e-4e09-a855-b88911cc9204 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:00.395611 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f828c766-79c1-4904-9fca-3694486a042d None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:00.422294 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:00.422523 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:00.429664 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a309159-ac32-4703-a0d3-6a5e6e426277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:00.430563 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a309159-ac32-4703-a0d3-6a5e6e426277 None admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:00.430997 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a309159-ac32-4703-a0d3-6a5e6e426277 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:00.431419 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a309159-ac32-4703-a0d3-6a5e6e426277 None admin] PATH_INFO: `/v3/groups` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:00.432222 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a309159-ac32-4703-a0d3-6a5e6e426277 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:00.433910 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a309159-ac32-4703-a0d3-6a5e6e426277 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:00.436463 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec404f3d-c55c-4c28-9916-1ef906765469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:00.437121 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec404f3d-c55c-4c28-9916-1ef906765469 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:00.437411 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec404f3d-c55c-4c28-9916-1ef906765469 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:00.437733 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec404f3d-c55c-4c28-9916-1ef906765469 None admin] PATH_INFO: `/v3/users/362c0984745e40f887648a4fe6ef2ea8` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:00.439135 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec404f3d-c55c-4c28-9916-1ef906765469 None admin] RBAC: Authorizing `identity:get_user(user_id=362c0984745e40f887648a4fe6ef2ea8)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:00.440564 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec404f3d-c55c-4c28-9916-1ef906765469 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:00.443008 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 157/307] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Jul 26 13:33:00 2022] GET /identity/v3/users/362c0984745e40f887648a4fe6ef2ea8 => generated 293 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:00.443669 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec404f3d-c55c-4c28-9916-1ef906765469 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:00.458118 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 151/308] 10.222.0.58 () {60 vars in 1134 bytes} [Tue Jul 26 13:33:00 2022] POST /identity/v3/groups => generated 229 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:33:00.459305 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a309159-ac32-4703-a0d3-6a5e6e426277 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:00.591777 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a309159-ac32-4703-a0d3-6a5e6e426277 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:00.613291 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec404f3d-c55c-4c28-9916-1ef906765469 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:00.649198 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061eed4a-8de7-44e7-b86f-6f5a8e7d9f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:00.650221 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061eed4a-8de7-44e7-b86f-6f5a8e7d9f85 None admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:00.650504 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061eed4a-8de7-44e7-b86f-6f5a8e7d9f85 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:00.650785 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061eed4a-8de7-44e7-b86f-6f5a8e7d9f85 None admin] PATH_INFO: `/v3/system/users/362c0984745e40f887648a4fe6ef2ea8/roles/759f389b221049c4b01faf3b984cd8d7` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:00.652903 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061eed4a-8de7-44e7-b86f-6f5a8e7d9f85 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=362c0984745e40f887648a4fe6ef2ea8, role_id=759f389b221049c4b01faf3b984cd8d7)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:00.654994 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061eed4a-8de7-44e7-b86f-6f5a8e7d9f85 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:00.661138 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3c7cbb-b9c2-4554-bd4e-569122a6cd99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:00.662494 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3c7cbb-b9c2-4554-bd4e-569122a6cd99 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:00.662494 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3c7cbb-b9c2-4554-bd4e-569122a6cd99 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:00.662494 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3c7cbb-b9c2-4554-bd4e-569122a6cd99 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:00.663465 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3c7cbb-b9c2-4554-bd4e-569122a6cd99 None admin] RBAC: Adding query filter params (role.id=39ccb3797eb542529c79507664d04b53, user.id=a789c6c8fc6f48ac9a5b9c1295b5fea1, scope.system=all) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:00.663873 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3c7cbb-b9c2-4554-bd4e-569122a6cd99 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:00.665725 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3c7cbb-b9c2-4554-bd4e-569122a6cd99 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:00.668755 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 152/309] 10.222.0.58 () {58 vars in 1334 bytes} [Tue Jul 26 13:33:00 2022] PUT /identity/v3/system/users/362c0984745e40f887648a4fe6ef2ea8/roles/759f389b221049c4b01faf3b984cd8d7 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:33:00.672556 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 158/310] 10.222.0.58 () {56 vars in 1308 bytes} [Tue Jul 26 13:33:00 2022] GET /identity/v3/role_assignments?user.id=a789c6c8fc6f48ac9a5b9c1295b5fea1&scope.system=all&role.id=39ccb3797eb542529c79507664d04b53 => generated 226 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:01.264540 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68ecdd1-3ea2-4ac8-9c59-b0ea2fd3a2f0 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:01.264824 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68ecdd1-3ea2-4ac8-9c59-b0ea2fd3a2f0 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:01.265103 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68ecdd1-3ea2-4ac8-9c59-b0ea2fd3a2f0 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:01.266778 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 153/311] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:33:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:33:01.272322 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac68339-c8dc-4991-80fc-4aa470763199 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:01.272610 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac68339-c8dc-4991-80fc-4aa470763199 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:01.272880 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac68339-c8dc-4991-80fc-4aa470763199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:01.291457 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-5ac68339-c8dc-4991-80fc-4aa470763199 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:33:01.293080 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-5ac68339-c8dc-4991-80fc-4aa470763199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:33:01.353661 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:01.377104 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 159/312] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:33:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 13:33:01.402181 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68ecdd1-3ea2-4ac8-9c59-b0ea2fd3a2f0 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:01.470359 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:01.522468 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68ecdd1-3ea2-4ac8-9c59-b0ea2fd3a2f0 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:01.563571 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f00579-4aee-4271-9554-339f67f1caab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:01.564139 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f00579-4aee-4271-9554-339f67f1caab None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:01.564421 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f00579-4aee-4271-9554-339f67f1caab None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:01.564702 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f00579-4aee-4271-9554-339f67f1caab None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:01.575983 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f00579-4aee-4271-9554-339f67f1caab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:33:01.581616 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-b3f00579-4aee-4271-9554-339f67f1caab None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 13:33:01.581616 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:01.581616 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:01.581616 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:01.581616 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:01.581616 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:01.581616 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:01.581616 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:01.581616 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:01.581616 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:01.581616 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:01.581616 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:01.581616 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:33:01.581616 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:33:01.581616 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:33:01.581616 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:33:01.582522 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:33:01.582522 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:33:01.582522 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:33:01.582522 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:33:01.582522 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:01.582522 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:01.582522 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:01.582522 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:01.582522 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:01.582522 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:01.582522 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:01.582522 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:01.582522 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:01.582522 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:01.582522 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:01.582522 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:01.583306 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:01.583306 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:01.583306 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:01.583306 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:01.583306 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:33:01.583306 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:33:01.583306 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:33:01.583306 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:33:01.583306 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:33:01.583306 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:33:01.583306 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 13:33:01.583306 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:33:01.583306 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 154/313] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Jul 26 13:33:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:01.590988 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac68339-c8dc-4991-80fc-4aa470763199 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:01.697580 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac68339-c8dc-4991-80fc-4aa470763199 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:01.737451 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58394d82-c7a8-4bd4-a43b-9e09aa9e791d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:01.738201 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58394d82-c7a8-4bd4-a43b-9e09aa9e791d None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:01.738492 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58394d82-c7a8-4bd4-a43b-9e09aa9e791d None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:01.738770 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58394d82-c7a8-4bd4-a43b-9e09aa9e791d None admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:01.739511 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58394d82-c7a8-4bd4-a43b-9e09aa9e791d None admin] RBAC: Adding query filter params (name=reader) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:01.739823 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58394d82-c7a8-4bd4-a43b-9e09aa9e791d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:01.741292 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58394d82-c7a8-4bd4-a43b-9e09aa9e791d None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:01.751577 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 160/314] 10.222.0.58 () {56 vars in 1101 bytes} [Tue Jul 26 13:33:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:01.757101 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f00579-4aee-4271-9554-339f67f1caab None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:01.876285 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f00579-4aee-4271-9554-339f67f1caab None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:01.893247 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:01.920726 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89f7194-8e37-434c-bd29-d973d3f98550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:01.921564 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89f7194-8e37-434c-bd29-d973d3f98550 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:01.921952 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89f7194-8e37-434c-bd29-d973d3f98550 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:01.922332 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89f7194-8e37-434c-bd29-d973d3f98550 None admin] PATH_INFO: `/v3/users/1e91c3827fba42fe81d50be2d6525528` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:01.923976 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89f7194-8e37-434c-bd29-d973d3f98550 None admin] RBAC: Authorizing `identity:get_user(user_id=1e91c3827fba42fe81d50be2d6525528)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:01.926297 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89f7194-8e37-434c-bd29-d973d3f98550 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:01.928578 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 155/315] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Jul 26 13:33:01 2022] GET /identity/v3/users/1e91c3827fba42fe81d50be2d6525528 => generated 297 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:01.933849 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58394d82-c7a8-4bd4-a43b-9e09aa9e791d None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:01.983328 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:02.044878 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58394d82-c7a8-4bd4-a43b-9e09aa9e791d None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:02.086852 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf8030b-67b6-4f1c-88f4-30cc3ae44e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:02.087876 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf8030b-67b6-4f1c-88f4-30cc3ae44e99 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:02.088360 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf8030b-67b6-4f1c-88f4-30cc3ae44e99 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:02.088835 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf8030b-67b6-4f1c-88f4-30cc3ae44e99 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:02.160575 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf8030b-67b6-4f1c-88f4-30cc3ae44e99 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:02.162356 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf8030b-67b6-4f1c-88f4-30cc3ae44e99 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:02.211475 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:02.269219 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 161/316] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Jul 26 13:33:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 13:33:02.281510 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89f7194-8e37-434c-bd29-d973d3f98550 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:02.395425 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89f7194-8e37-434c-bd29-d973d3f98550 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:02.437564 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16f8e23-b562-48a4-ba92-472f5a5fc9e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:02.438185 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16f8e23-b562-48a4-ba92-472f5a5fc9e8 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:02.438421 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16f8e23-b562-48a4-ba92-472f5a5fc9e8 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:02.438764 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16f8e23-b562-48a4-ba92-472f5a5fc9e8 None admin] PATH_INFO: `/v3/projects/6ad88492b683452a81ab7150c22b665e` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:02.440038 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16f8e23-b562-48a4-ba92-472f5a5fc9e8 None admin] RBAC: Authorizing `identity:get_project(project_id=6ad88492b683452a81ab7150c22b665e)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:02.442516 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16f8e23-b562-48a4-ba92-472f5a5fc9e8 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:02.444827 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 156/317] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Jul 26 13:33:02 2022] GET /identity/v3/projects/6ad88492b683452a81ab7150c22b665e => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:02.451804 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf8030b-67b6-4f1c-88f4-30cc3ae44e99 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:02.565068 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf8030b-67b6-4f1c-88f4-30cc3ae44e99 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:02.603162 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7594b2-9e2c-48c0-8f06-a314c6c89f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:02.604194 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7594b2-9e2c-48c0-8f06-a314c6c89f16 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:02.604704 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7594b2-9e2c-48c0-8f06-a314c6c89f16 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:02.605735 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7594b2-9e2c-48c0-8f06-a314c6c89f16 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:02.606737 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7594b2-9e2c-48c0-8f06-a314c6c89f16 None admin] RBAC: Adding query filter params (role.id=39ccb3797eb542529c79507664d04b53, scope.project.id=6ad88492b683452a81ab7150c22b665e, user.id=1e91c3827fba42fe81d50be2d6525528) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:02.607240 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7594b2-9e2c-48c0-8f06-a314c6c89f16 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:02.609909 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7594b2-9e2c-48c0-8f06-a314c6c89f16 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:02.618191 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 162/318] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Jul 26 13:33:02 2022] GET /identity/v3/role_assignments?user.id=1e91c3827fba42fe81d50be2d6525528&scope.project.id=6ad88492b683452a81ab7150c22b665e&role.id=39ccb3797eb542529c79507664d04b53 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:02.726624 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66092c60-0113-4069-a0e0-335a990186dc None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:02.726624 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66092c60-0113-4069-a0e0-335a990186dc None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:02.726624 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66092c60-0113-4069-a0e0-335a990186dc None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:02.727272 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 157/319] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:33:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:33:02.733846 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4860c7f1-812b-42ea-ab26-1d0f5fdcd875 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:02.734460 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4860c7f1-812b-42ea-ab26-1d0f5fdcd875 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:02.734932 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4860c7f1-812b-42ea-ab26-1d0f5fdcd875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:02.755370 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-4860c7f1-812b-42ea-ab26-1d0f5fdcd875 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:33:02.756942 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-4860c7f1-812b-42ea-ab26-1d0f5fdcd875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:33:02.760637 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:02.839691 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 163/320] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:33:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 13:33:02.842785 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf15e96b-81cb-466e-9ffb-2d425a23ef9e None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:02.843073 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf15e96b-81cb-466e-9ffb-2d425a23ef9e None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:02.843359 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf15e96b-81cb-466e-9ffb-2d425a23ef9e None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:02.844710 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 158/321] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:33:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:33:02.850906 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49be9a1-e900-43b8-aa25-00b699b7b12c None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:02.851200 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49be9a1-e900-43b8-aa25-00b699b7b12c None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:02.851482 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49be9a1-e900-43b8-aa25-00b699b7b12c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:02.863298 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf15e96b-81cb-466e-9ffb-2d425a23ef9e None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:02.874624 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-f49be9a1-e900-43b8-aa25-00b699b7b12c None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:33:02.875858 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-f49be9a1-e900-43b8-aa25-00b699b7b12c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:33:02.924876 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:02.950886 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 164/322] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:33:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 13:33:02.955099 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97e1b63-5ee8-4c9c-bf96-6f8013eed343 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:02.955388 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97e1b63-5ee8-4c9c-bf96-6f8013eed343 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:02.955669 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97e1b63-5ee8-4c9c-bf96-6f8013eed343 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:02.957156 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 165/323] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:33:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:33:02.962674 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86e3d9b-6201-4223-bca9-6c88a657eb25 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:02.962963 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86e3d9b-6201-4223-bca9-6c88a657eb25 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:02.963239 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86e3d9b-6201-4223-bca9-6c88a657eb25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:02.981175 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf15e96b-81cb-466e-9ffb-2d425a23ef9e None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:02.983137 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-c86e3d9b-6201-4223-bca9-6c88a657eb25 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:33:02.984355 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-c86e3d9b-6201-4223-bca9-6c88a657eb25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:33:03.014895 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53bf796-eb9a-4195-94b1-3c7b186caa06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:03.015542 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53bf796-eb9a-4195-94b1-3c7b186caa06 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:03.015825 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53bf796-eb9a-4195-94b1-3c7b186caa06 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:03.016101 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53bf796-eb9a-4195-94b1-3c7b186caa06 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:03.017246 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53bf796-eb9a-4195-94b1-3c7b186caa06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:03.018862 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53bf796-eb9a-4195-94b1-3c7b186caa06 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:03.025246 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 159/324] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:33:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:33:03.025246 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53bf796-eb9a-4195-94b1-3c7b186caa06 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:03.054177 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:03.065049 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 166/325] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:33:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 13:33:03.066234 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86e3d9b-6201-4223-bca9-6c88a657eb25 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:03.143443 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53bf796-eb9a-4195-94b1-3c7b186caa06 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:03.161394 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:03.175958 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86e3d9b-6201-4223-bca9-6c88a657eb25 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:03.181583 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22c8571-133d-40cc-9ab7-f3ac3b04db75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:03.182234 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22c8571-133d-40cc-9ab7-f3ac3b04db75 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:03.182520 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22c8571-133d-40cc-9ab7-f3ac3b04db75 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:03.182795 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22c8571-133d-40cc-9ab7-f3ac3b04db75 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:03.193572 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22c8571-133d-40cc-9ab7-f3ac3b04db75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:33:03.199213 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-c22c8571-133d-40cc-9ab7-f3ac3b04db75 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 13:33:03.199213 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:03.199213 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:03.199213 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:03.199213 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:03.199213 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:03.199213 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:03.199213 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:03.199213 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:03.199213 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:03.199213 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:03.199213 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:03.199213 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:33:03.199213 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:33:03.199213 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:33:03.199213 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:33:03.200148 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:33:03.200148 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:33:03.200148 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:33:03.200148 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:33:03.200148 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:03.200148 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:03.200148 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:03.200148 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:03.200148 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:03.200148 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:03.200148 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:03.200148 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:03.200148 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:03.200148 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:03.200148 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:03.200148 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:03.200971 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:03.200971 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:03.200971 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:03.200971 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:03.200971 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:33:03.200971 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:33:03.200971 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:33:03.200971 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:33:03.200971 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:33:03.200971 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:33:03.200971 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 13:33:03.200971 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:33:03.200971 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 160/326] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Jul 26 13:33:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:03.201642 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22c8571-133d-40cc-9ab7-f3ac3b04db75 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:03.218500 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba93fb9-c85f-4ba3-9324-e5ba8e182a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:03.219148 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba93fb9-c85f-4ba3-9324-e5ba8e182a47 None admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:03.219429 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba93fb9-c85f-4ba3-9324-e5ba8e182a47 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:03.219701 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba93fb9-c85f-4ba3-9324-e5ba8e182a47 None admin] PATH_INFO: `/v3/groups` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:03.220371 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba93fb9-c85f-4ba3-9324-e5ba8e182a47 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:03.221827 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba93fb9-c85f-4ba3-9324-e5ba8e182a47 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:03.241397 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 167/327] 10.222.0.58 () {60 vars in 1134 bytes} [Tue Jul 26 13:33:03 2022] POST /identity/v3/groups => generated 226 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:33:03.242227 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba93fb9-c85f-4ba3-9324-e5ba8e182a47 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:03.316611 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:03.334465 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22c8571-133d-40cc-9ab7-f3ac3b04db75 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:03.376182 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba93fb9-c85f-4ba3-9324-e5ba8e182a47 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:03.388677 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447ba276-f6c4-49bc-b17f-4958f4b7ae8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:03.389598 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447ba276-f6c4-49bc-b17f-4958f4b7ae8d None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:03.389933 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447ba276-f6c4-49bc-b17f-4958f4b7ae8d None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:03.390212 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447ba276-f6c4-49bc-b17f-4958f4b7ae8d None admin] PATH_INFO: `/v3/roles/member` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:03.406572 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447ba276-f6c4-49bc-b17f-4958f4b7ae8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:33:03.412073 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-447ba276-f6c4-49bc-b17f-4958f4b7ae8d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 13:33:03.412073 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:03.412073 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:03.412073 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:03.412073 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:03.412073 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:03.412073 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:03.412073 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:03.412073 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:03.412073 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:03.412073 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:03.412073 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:03.412073 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:33:03.412073 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:33:03.412073 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:33:03.412073 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:33:03.412957 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:33:03.412957 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:33:03.412957 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:33:03.412957 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:33:03.412957 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:03.412957 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:03.412957 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:03.412957 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:03.412957 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:03.412957 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:03.412957 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:03.412957 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:03.412957 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:03.412957 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:03.412957 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:03.412957 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:03.413868 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:03.413868 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:03.413868 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:03.413868 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:03.413868 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:33:03.413868 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:33:03.413868 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:33:03.413868 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:33:03.413868 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:33:03.413868 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:33:03.413868 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 13:33:03.413868 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:33:03.413868 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 161/328] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Jul 26 13:33:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:03.419811 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447ba276-f6c4-49bc-b17f-4958f4b7ae8d None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:03.428171 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecaf127-3c51-467e-b7f2-818061fc5002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:03.428829 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecaf127-3c51-467e-b7f2-818061fc5002 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:03.429122 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecaf127-3c51-467e-b7f2-818061fc5002 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:03.429379 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecaf127-3c51-467e-b7f2-818061fc5002 None admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:03.430486 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecaf127-3c51-467e-b7f2-818061fc5002 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:03.430486 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecaf127-3c51-467e-b7f2-818061fc5002 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:03.432214 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecaf127-3c51-467e-b7f2-818061fc5002 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:03.440824 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 168/329] 10.222.0.58 () {56 vars in 1101 bytes} [Tue Jul 26 13:33:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:03.447267 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecaf127-3c51-467e-b7f2-818061fc5002 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:03.474580 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:03.557146 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447ba276-f6c4-49bc-b17f-4958f4b7ae8d None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:03.575326 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecaf127-3c51-467e-b7f2-818061fc5002 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:03.605369 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4aff84-f40c-49b6-a730-cdd953ebdfcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:03.606221 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4aff84-f40c-49b6-a730-cdd953ebdfcb None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:03.606513 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4aff84-f40c-49b6-a730-cdd953ebdfcb None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:03.606792 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4aff84-f40c-49b6-a730-cdd953ebdfcb None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:03.607535 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4aff84-f40c-49b6-a730-cdd953ebdfcb None admin] RBAC: Adding query filter params (name=member) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:03.607666 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:03.607849 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4aff84-f40c-49b6-a730-cdd953ebdfcb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:03.609266 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4aff84-f40c-49b6-a730-cdd953ebdfcb None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:03.621328 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 162/330] 10.222.0.58 () {56 vars in 1101 bytes} [Tue Jul 26 13:33:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:33:03.628484 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4aff84-f40c-49b6-a730-cdd953ebdfcb None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:03.633992 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b2192c-d70b-40c2-b0fb-331eb5b3dcd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:03.634925 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b2192c-d70b-40c2-b0fb-331eb5b3dcd5 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:03.635394 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b2192c-d70b-40c2-b0fb-331eb5b3dcd5 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:03.635668 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b2192c-d70b-40c2-b0fb-331eb5b3dcd5 None admin] PATH_INFO: `/v3/users/a789c6c8fc6f48ac9a5b9c1295b5fea1` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:03.637839 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b2192c-d70b-40c2-b0fb-331eb5b3dcd5 None admin] RBAC: Authorizing `identity:get_user(user_id=a789c6c8fc6f48ac9a5b9c1295b5fea1)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:03.639317 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b2192c-d70b-40c2-b0fb-331eb5b3dcd5 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:03.641311 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 169/331] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Jul 26 13:33:03 2022] GET /identity/v3/users/a789c6c8fc6f48ac9a5b9c1295b5fea1 => generated 293 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:33:03.648747 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b2192c-d70b-40c2-b0fb-331eb5b3dcd5 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:03.767460 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:03.772020 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4aff84-f40c-49b6-a730-cdd953ebdfcb None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:03.791047 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b2192c-d70b-40c2-b0fb-331eb5b3dcd5 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:03.817805 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0689b6b8-deb8-429b-b603-b17b4edb456f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:03.818664 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0689b6b8-deb8-429b-b603-b17b4edb456f None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:03.818950 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0689b6b8-deb8-429b-b603-b17b4edb456f None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:03.819266 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0689b6b8-deb8-429b-b603-b17b4edb456f None admin] PATH_INFO: `/v3/users/362c0984745e40f887648a4fe6ef2ea8` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:03.821223 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0689b6b8-deb8-429b-b603-b17b4edb456f None admin] RBAC: Authorizing `identity:get_user(user_id=362c0984745e40f887648a4fe6ef2ea8)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:03.823242 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0689b6b8-deb8-429b-b603-b17b4edb456f None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:03.826500 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 163/332] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Jul 26 13:33:03 2022] GET /identity/v3/users/362c0984745e40f887648a4fe6ef2ea8 => generated 293 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:33:03.832134 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26591548-261c-4fcc-bdc9-9386d63a9793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:03.832827 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26591548-261c-4fcc-bdc9-9386d63a9793 None admin] REQUEST_METHOD: `PUT` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:03.833108 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26591548-261c-4fcc-bdc9-9386d63a9793 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:03.833386 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26591548-261c-4fcc-bdc9-9386d63a9793 None admin] PATH_INFO: `/v3/system/users/a789c6c8fc6f48ac9a5b9c1295b5fea1/roles/39ccb3797eb542529c79507664d04b53` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:03.834441 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0689b6b8-deb8-429b-b603-b17b4edb456f None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:03.836627 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26591548-261c-4fcc-bdc9-9386d63a9793 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=a789c6c8fc6f48ac9a5b9c1295b5fea1, role_id=39ccb3797eb542529c79507664d04b53)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:03.838089 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26591548-261c-4fcc-bdc9-9386d63a9793 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:03.849581 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 170/333] 10.222.0.58 () {58 vars in 1334 bytes} [Tue Jul 26 13:33:03 2022] PUT /identity/v3/system/users/a789c6c8fc6f48ac9a5b9c1295b5fea1/roles/39ccb3797eb542529c79507664d04b53 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:33:03.989958 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0689b6b8-deb8-429b-b603-b17b4edb456f None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:04.035654 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5d4217-74d3-4731-a1ba-bd13e678d0c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:04.036294 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5d4217-74d3-4731-a1ba-bd13e678d0c3 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:04.036576 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5d4217-74d3-4731-a1ba-bd13e678d0c3 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:04.036850 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5d4217-74d3-4731-a1ba-bd13e678d0c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:04.037683 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5d4217-74d3-4731-a1ba-bd13e678d0c3 None admin] RBAC: Adding query filter params (role.id=759f389b221049c4b01faf3b984cd8d7, user.id=362c0984745e40f887648a4fe6ef2ea8, scope.system=all) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:04.037997 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5d4217-74d3-4731-a1ba-bd13e678d0c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:04.039965 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5d4217-74d3-4731-a1ba-bd13e678d0c3 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:04.056137 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 164/334] 10.222.0.58 () {56 vars in 1308 bytes} [Tue Jul 26 13:33:03 2022] GET /identity/v3/role_assignments?user.id=362c0984745e40f887648a4fe6ef2ea8&scope.system=all&role.id=759f389b221049c4b01faf3b984cd8d7 => generated 510 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:33:05.513521 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78298e38-a127-49c2-b426-676c68bbecc8 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:05.514566 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78298e38-a127-49c2-b426-676c68bbecc8 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:05.515075 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78298e38-a127-49c2-b426-676c68bbecc8 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:05.516985 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 171/335] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:33:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:33:05.522652 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2992984b-5d37-4c17-893a-03f67b090184 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:05.523181 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2992984b-5d37-4c17-893a-03f67b090184 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:05.523257 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2992984b-5d37-4c17-893a-03f67b090184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:05.550875 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-2992984b-5d37-4c17-893a-03f67b090184 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:33:05.552107 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-2992984b-5d37-4c17-893a-03f67b090184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:33:05.577884 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:05.641565 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 165/336] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:33:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 13:33:05.680423 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78298e38-a127-49c2-b426-676c68bbecc8 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:05.772737 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:05.838908 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78298e38-a127-49c2-b426-676c68bbecc8 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:05.889777 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd59b46-b6fe-4118-9908-a64181bd5200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:05.890819 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd59b46-b6fe-4118-9908-a64181bd5200 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:05.891306 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd59b46-b6fe-4118-9908-a64181bd5200 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:05.891806 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd59b46-b6fe-4118-9908-a64181bd5200 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:05.915711 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd59b46-b6fe-4118-9908-a64181bd5200 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:33:05.921905 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-2bd59b46-b6fe-4118-9908-a64181bd5200 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 13:33:05.921905 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:05.921905 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:05.921905 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:05.921905 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:05.921905 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:05.921905 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:05.921905 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:05.921905 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:05.921905 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:05.921905 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:05.921905 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:05.921905 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:33:05.921905 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:33:05.921905 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:33:05.921905 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:33:05.922791 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:33:05.922791 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:33:05.922791 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:33:05.922791 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:33:05.922791 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:05.922791 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:05.922791 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:05.922791 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:05.922791 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:05.922791 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:05.922791 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:05.922791 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:05.922791 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:05.922791 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:05.922791 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:05.922791 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:05.923665 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:05.923665 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:05.923665 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:05.923665 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:05.923665 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:33:05.923665 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:33:05.923665 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:33:05.923665 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:33:05.923665 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:33:05.923665 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:33:05.923665 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 13:33:05.923665 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:33:05.930971 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 172/337] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Jul 26 13:33:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:05.942667 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2992984b-5d37-4c17-893a-03f67b090184 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:06.088997 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2992984b-5d37-4c17-893a-03f67b090184 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:06.133040 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4144c67a-9c0b-4a07-853e-d065ae076037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:06.133707 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4144c67a-9c0b-4a07-853e-d065ae076037 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:06.134085 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4144c67a-9c0b-4a07-853e-d065ae076037 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:06.134372 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4144c67a-9c0b-4a07-853e-d065ae076037 None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:06.135111 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4144c67a-9c0b-4a07-853e-d065ae076037 None admin] RBAC: Adding query filter params (name=member) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:06.135420 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4144c67a-9c0b-4a07-853e-d065ae076037 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:06.137368 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4144c67a-9c0b-4a07-853e-d065ae076037 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:06.151100 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 166/338] 10.222.0.58 () {56 vars in 1101 bytes} [Tue Jul 26 13:33:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:06.152501 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd59b46-b6fe-4118-9908-a64181bd5200 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:06.282493 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd59b46-b6fe-4118-9908-a64181bd5200 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:06.298582 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:06.328917 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71cb501-2c9c-4edc-ae0f-d387f7cb2477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:06.329961 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71cb501-2c9c-4edc-ae0f-d387f7cb2477 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:06.330445 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71cb501-2c9c-4edc-ae0f-d387f7cb2477 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:06.330957 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71cb501-2c9c-4edc-ae0f-d387f7cb2477 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:06.424247 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71cb501-2c9c-4edc-ae0f-d387f7cb2477 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:06.426551 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71cb501-2c9c-4edc-ae0f-d387f7cb2477 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:06.538110 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 173/339] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Jul 26 13:33:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 13:33:06.544321 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4144c67a-9c0b-4a07-853e-d065ae076037 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:06.584299 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:06.586922 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2218740-c9d1-4a15-a7c3-3dadd9232b4c None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:06.587444 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2218740-c9d1-4a15-a7c3-3dadd9232b4c None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:06.587882 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2218740-c9d1-4a15-a7c3-3dadd9232b4c None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:06.594391 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 174/340] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:33:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 13:33:06.596836 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b589bc-69c4-4ced-9556-e108e9c76e00 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:06.597382 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b589bc-69c4-4ced-9556-e108e9c76e00 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:06.598001 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b589bc-69c4-4ced-9556-e108e9c76e00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:06.619340 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-82b589bc-69c4-4ced-9556-e108e9c76e00 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:33:06.621680 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-82b589bc-69c4-4ced-9556-e108e9c76e00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:33:06.633117 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:06.701586 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 175/341] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:33:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 13:33:06.703387 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6be32a-5284-4939-9030-8e5f648cdd32 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:06.703878 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6be32a-5284-4939-9030-8e5f648cdd32 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:06.704347 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6be32a-5284-4939-9030-8e5f648cdd32 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:06.708699 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 176/342] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:33:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:33:06.716053 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7d3c47-1519-4d7c-9344-4e92719e4bfe None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:06.716883 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7d3c47-1519-4d7c-9344-4e92719e4bfe None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:06.717456 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7d3c47-1519-4d7c-9344-4e92719e4bfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:06.745517 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4144c67a-9c0b-4a07-853e-d065ae076037 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:06.748794 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-dc7d3c47-1519-4d7c-9344-4e92719e4bfe None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:33:06.750349 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-dc7d3c47-1519-4d7c-9344-4e92719e4bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:33:06.826438 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08df36d7-02be-4bfd-9f5a-8b19693bca85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:06.827109 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08df36d7-02be-4bfd-9f5a-8b19693bca85 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:06.827506 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08df36d7-02be-4bfd-9f5a-8b19693bca85 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:06.827796 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08df36d7-02be-4bfd-9f5a-8b19693bca85 None admin] PATH_INFO: `/v3/projects/afb9ca9466054e008e3b052866da363d` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:06.834525 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08df36d7-02be-4bfd-9f5a-8b19693bca85 None admin] RBAC: Authorizing `identity:get_project(project_id=afb9ca9466054e008e3b052866da363d)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:06.835933 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08df36d7-02be-4bfd-9f5a-8b19693bca85 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:06.838049 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 167/343] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Jul 26 13:33:06 2022] GET /identity/v3/projects/afb9ca9466054e008e3b052866da363d => generated 298 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:06.838965 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08df36d7-02be-4bfd-9f5a-8b19693bca85 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:06.846523 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 177/344] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:33:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 13:33:06.860174 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7d3c47-1519-4d7c-9344-4e92719e4bfe None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:06.958530 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:06.994114 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7d3c47-1519-4d7c-9344-4e92719e4bfe None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:07.009612 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:07.032542 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08df36d7-02be-4bfd-9f5a-8b19693bca85 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:07.049649 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491a850e-9d23-45bb-a810-2af5bc0c7c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:07.049649 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491a850e-9d23-45bb-a810-2af5bc0c7c36 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:07.049649 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491a850e-9d23-45bb-a810-2af5bc0c7c36 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:07.049649 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491a850e-9d23-45bb-a810-2af5bc0c7c36 None admin] PATH_INFO: `/v3/groups/04331d538f1c4d479d659675e703da04` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:07.055003 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491a850e-9d23-45bb-a810-2af5bc0c7c36 None admin] RBAC: Authorizing `identity:get_group(group_id=04331d538f1c4d479d659675e703da04)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:07.056563 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491a850e-9d23-45bb-a810-2af5bc0c7c36 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:07.058490 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 178/345] 10.222.0.58 () {56 vars in 1180 bytes} [Tue Jul 26 13:33:06 2022] GET /identity/v3/groups/04331d538f1c4d479d659675e703da04 => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:33:07.059334 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491a850e-9d23-45bb-a810-2af5bc0c7c36 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:07.086188 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ddfb7c-3984-47e1-ae8f-a50ef6ea8674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:07.087183 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ddfb7c-3984-47e1-ae8f-a50ef6ea8674 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:07.087688 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ddfb7c-3984-47e1-ae8f-a50ef6ea8674 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:07.088163 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ddfb7c-3984-47e1-ae8f-a50ef6ea8674 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:07.105791 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ddfb7c-3984-47e1-ae8f-a50ef6ea8674 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:33:07.115979 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-d4ddfb7c-3984-47e1-ae8f-a50ef6ea8674 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 13:33:07.115979 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:07.115979 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:07.115979 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:07.115979 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:07.115979 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:07.115979 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:07.115979 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:07.115979 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:07.115979 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:07.115979 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:07.115979 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:07.115979 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:33:07.115979 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:33:07.115979 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:33:07.115979 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:33:07.116927 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:33:07.116927 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:33:07.116927 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:33:07.116927 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:33:07.116927 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:07.116927 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:07.116927 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:07.116927 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:07.116927 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:07.116927 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:07.116927 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:07.116927 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:07.116927 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:07.116927 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:07.116927 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:07.116927 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:07.117751 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:07.117751 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:07.117751 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:07.117751 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:07.117751 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:33:07.117751 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:33:07.117751 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:33:07.117751 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:33:07.117751 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:33:07.117751 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:33:07.117751 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 13:33:07.117751 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:33:07.120687 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 168/346] 10.222.0.58 () {56 vars in 1114 bytes} [Tue Jul 26 13:33:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:07.122227 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f48dd4e-78f6-4089-a1a6-d6e285774aac None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:07.122722 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f48dd4e-78f6-4089-a1a6-d6e285774aac None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:07.123210 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f48dd4e-78f6-4089-a1a6-d6e285774aac None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:07.129197 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 169/347] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:33:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:33:07.130405 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f48dd4e-78f6-4089-a1a6-d6e285774aac None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:07.187792 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491a850e-9d23-45bb-a810-2af5bc0c7c36 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:07.231603 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57601002-95fe-4235-b475-3e3de8bb42ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:07.232850 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57601002-95fe-4235-b475-3e3de8bb42ca None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:07.233348 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57601002-95fe-4235-b475-3e3de8bb42ca None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:07.233698 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57601002-95fe-4235-b475-3e3de8bb42ca None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:07.243808 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:07.250734 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57601002-95fe-4235-b475-3e3de8bb42ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:33:07.253993 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f48dd4e-78f6-4089-a1a6-d6e285774aac None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:07.259176 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-57601002-95fe-4235-b475-3e3de8bb42ca None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 13:33:07.259176 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:07.259176 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:07.259176 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:07.259176 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:07.259176 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:07.259176 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:07.259176 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:07.259176 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:07.259176 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:07.259176 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:07.259176 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:07.259176 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:33:07.259176 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:33:07.259176 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:33:07.259176 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:33:07.260328 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:33:07.260328 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:33:07.260328 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:33:07.260328 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:33:07.260328 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:07.260328 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:07.260328 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:07.260328 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:07.260328 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:07.260328 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:07.260328 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:07.260328 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:07.260328 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:07.260328 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:07.260328 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:07.260328 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:07.262482 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:07.262482 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:07.262482 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:07.262482 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:07.262482 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:33:07.262482 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:33:07.262482 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:33:07.262482 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:33:07.262482 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:33:07.262482 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:33:07.262482 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 13:33:07.262482 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:33:07.266190 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 179/348] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Jul 26 13:33:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:07.268136 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b7a1bd-ceaa-47a9-a1f6-7b7fbe111874 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:07.273509 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b7a1bd-ceaa-47a9-a1f6-7b7fbe111874 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:07.274555 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b7a1bd-ceaa-47a9-a1f6-7b7fbe111874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:07.295250 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9af7a40-370e-427c-981e-f1f9d088fe2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:07.295902 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9af7a40-370e-427c-981e-f1f9d088fe2d None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:07.296182 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9af7a40-370e-427c-981e-f1f9d088fe2d None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:07.296456 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9af7a40-370e-427c-981e-f1f9d088fe2d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:07.297313 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9af7a40-370e-427c-981e-f1f9d088fe2d None admin] RBAC: Adding query filter params (group.id=04331d538f1c4d479d659675e703da04, role.id=759f389b221049c4b01faf3b984cd8d7, scope.project.id=afb9ca9466054e008e3b052866da363d) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:07.297659 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9af7a40-370e-427c-981e-f1f9d088fe2d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:07.299185 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9af7a40-370e-427c-981e-f1f9d088fe2d None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:07.301294 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-50b7a1bd-ceaa-47a9-a1f6-7b7fbe111874 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:33:07.302873 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-50b7a1bd-ceaa-47a9-a1f6-7b7fbe111874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:33:07.306137 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 170/349] 10.222.0.58 () {56 vars in 1376 bytes} [Tue Jul 26 13:33:07 2022] GET /identity/v3/role_assignments?group.id=04331d538f1c4d479d659675e703da04&scope.project.id=afb9ca9466054e008e3b052866da363d&role.id=759f389b221049c4b01faf3b984cd8d7 => generated 260 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:07.306792 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9af7a40-370e-427c-981e-f1f9d088fe2d None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:07.320800 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:07.411886 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 180/350] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:33:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 13:33:07.413055 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b7a1bd-ceaa-47a9-a1f6-7b7fbe111874 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:07.425715 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9af7a40-370e-427c-981e-f1f9d088fe2d None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:07.470703 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285ac4a6-6be9-47c3-a46f-a80b24fab139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:07.471339 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285ac4a6-6be9-47c3-a46f-a80b24fab139 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:07.471621 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285ac4a6-6be9-47c3-a46f-a80b24fab139 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:07.471896 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285ac4a6-6be9-47c3-a46f-a80b24fab139 None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:07.472627 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285ac4a6-6be9-47c3-a46f-a80b24fab139 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:07.472936 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285ac4a6-6be9-47c3-a46f-a80b24fab139 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:07.474362 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285ac4a6-6be9-47c3-a46f-a80b24fab139 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:07.485367 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 171/351] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Jul 26 13:33:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:07.486326 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285ac4a6-6be9-47c3-a46f-a80b24fab139 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:07.549047 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b7a1bd-ceaa-47a9-a1f6-7b7fbe111874 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:07.577921 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:07.590649 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:07.603635 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285ac4a6-6be9-47c3-a46f-a80b24fab139 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:07.604091 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6b40c2-311d-4f21-9912-71d872314db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:07.605259 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6b40c2-311d-4f21-9912-71d872314db6 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:07.605767 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6b40c2-311d-4f21-9912-71d872314db6 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:07.606246 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6b40c2-311d-4f21-9912-71d872314db6 None admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:07.607225 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6b40c2-311d-4f21-9912-71d872314db6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:07.607734 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6b40c2-311d-4f21-9912-71d872314db6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:07.611276 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6b40c2-311d-4f21-9912-71d872314db6 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:07.626351 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 181/352] 10.222.0.58 () {56 vars in 1101 bytes} [Tue Jul 26 13:33:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:07.627943 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6b40c2-311d-4f21-9912-71d872314db6 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:07.645777 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4ba18f-d4da-4564-be83-caf951a47682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:07.646464 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4ba18f-d4da-4564-be83-caf951a47682 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:07.646749 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4ba18f-d4da-4564-be83-caf951a47682 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:07.647022 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4ba18f-d4da-4564-be83-caf951a47682 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:07.660096 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4ba18f-d4da-4564-be83-caf951a47682 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:33:07.667128 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-6d4ba18f-d4da-4564-be83-caf951a47682 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 13:33:07.667128 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:07.667128 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:07.667128 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:07.667128 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:07.667128 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:07.667128 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:07.667128 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:07.667128 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:07.667128 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:07.667128 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:07.667128 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:07.667128 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:33:07.667128 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:33:07.667128 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:33:07.667128 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:33:07.668221 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:33:07.668221 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:33:07.668221 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:33:07.668221 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:33:07.668221 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:07.668221 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:07.668221 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:07.668221 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:07.668221 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:07.668221 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:07.668221 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:07.668221 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:07.668221 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:07.668221 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:07.668221 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:07.668221 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:07.668992 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:07.668992 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:07.668992 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:07.668992 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:07.668992 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:33:07.668992 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:33:07.668992 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:33:07.668992 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:33:07.668992 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:33:07.668992 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:33:07.668992 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 13:33:07.668992 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:33:07.668992 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 172/353] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Jul 26 13:33:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:07.669665 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4ba18f-d4da-4564-be83-caf951a47682 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:07.758725 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6b40c2-311d-4f21-9912-71d872314db6 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:07.803077 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e12ad8c-77ad-4ae6-9e8f-f9a7fc0fd61f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:07.804478 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e12ad8c-77ad-4ae6-9e8f-f9a7fc0fd61f None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:07.805220 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e12ad8c-77ad-4ae6-9e8f-f9a7fc0fd61f None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:07.805980 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e12ad8c-77ad-4ae6-9e8f-f9a7fc0fd61f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:07.826133 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4ba18f-d4da-4564-be83-caf951a47682 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:07.861919 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:07.867708 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a393fa5-e6ad-4103-aa7d-7927839555c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:07.868381 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a393fa5-e6ad-4103-aa7d-7927839555c8 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:07.868668 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a393fa5-e6ad-4103-aa7d-7927839555c8 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:07.871337 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a393fa5-e6ad-4103-aa7d-7927839555c8 None admin] PATH_INFO: `/v3/users/a789c6c8fc6f48ac9a5b9c1295b5fea1` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:07.872820 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a393fa5-e6ad-4103-aa7d-7927839555c8 None admin] RBAC: Authorizing `identity:get_user(user_id=a789c6c8fc6f48ac9a5b9c1295b5fea1)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:07.874488 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a393fa5-e6ad-4103-aa7d-7927839555c8 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:07.877304 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 173/354] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Jul 26 13:33:07 2022] GET /identity/v3/users/a789c6c8fc6f48ac9a5b9c1295b5fea1 => generated 293 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:07.878153 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a393fa5-e6ad-4103-aa7d-7927839555c8 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:07.893844 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:07.908464 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e12ad8c-77ad-4ae6-9e8f-f9a7fc0fd61f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:07.911565 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e12ad8c-77ad-4ae6-9e8f-f9a7fc0fd61f None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:08.005852 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a393fa5-e6ad-4103-aa7d-7927839555c8 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:08.047142 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a70f14b-500f-4cbf-995f-06947da138d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:08.047782 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a70f14b-500f-4cbf-995f-06947da138d6 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:08.048068 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a70f14b-500f-4cbf-995f-06947da138d6 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:08.048459 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a70f14b-500f-4cbf-995f-06947da138d6 None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:08.049312 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a70f14b-500f-4cbf-995f-06947da138d6 None admin] RBAC: Adding query filter params (name=member) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:08.053597 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a70f14b-500f-4cbf-995f-06947da138d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:08.053597 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a70f14b-500f-4cbf-995f-06947da138d6 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:08.062673 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 174/355] 10.222.0.58 () {56 vars in 1101 bytes} [Tue Jul 26 13:33:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:08.063550 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a70f14b-500f-4cbf-995f-06947da138d6 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:08.066483 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 182/356] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Jul 26 13:33:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 13:33:08.069538 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e12ad8c-77ad-4ae6-9e8f-f9a7fc0fd61f None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:08.149916 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:08.187411 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a70f14b-500f-4cbf-995f-06947da138d6 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:08.191274 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:08.202649 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e12ad8c-77ad-4ae6-9e8f-f9a7fc0fd61f None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:08.236127 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0551a4b5-7a81-4c19-93b0-12a91bdeecb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:08.236750 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0551a4b5-7a81-4c19-93b0-12a91bdeecb9 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:08.237037 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0551a4b5-7a81-4c19-93b0-12a91bdeecb9 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:08.237313 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0551a4b5-7a81-4c19-93b0-12a91bdeecb9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:08.238152 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0551a4b5-7a81-4c19-93b0-12a91bdeecb9 None admin] RBAC: Adding query filter params (role.id=39ccb3797eb542529c79507664d04b53, user.id=a789c6c8fc6f48ac9a5b9c1295b5fea1, scope.system=all) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:08.238464 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0551a4b5-7a81-4c19-93b0-12a91bdeecb9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:08.240056 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0551a4b5-7a81-4c19-93b0-12a91bdeecb9 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:08.254897 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac51450d-889b-4c2e-9cf8-cf7563f367ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:08.255539 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac51450d-889b-4c2e-9cf8-cf7563f367ae None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:08.255822 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac51450d-889b-4c2e-9cf8-cf7563f367ae None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:08.256147 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac51450d-889b-4c2e-9cf8-cf7563f367ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:08.266506 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 175/357] 10.222.0.58 () {56 vars in 1308 bytes} [Tue Jul 26 13:33:08 2022] GET /identity/v3/role_assignments?user.id=a789c6c8fc6f48ac9a5b9c1295b5fea1&scope.system=all&role.id=39ccb3797eb542529c79507664d04b53 => generated 510 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:08.267504 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0551a4b5-7a81-4c19-93b0-12a91bdeecb9 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:08.375699 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac51450d-889b-4c2e-9cf8-cf7563f367ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:08.378321 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac51450d-889b-4c2e-9cf8-cf7563f367ae None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:08.410835 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0551a4b5-7a81-4c19-93b0-12a91bdeecb9 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:08.435471 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:08.458324 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a230e10-8564-4f7e-9a45-bbafbc533943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:08.459695 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a230e10-8564-4f7e-9a45-bbafbc533943 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:08.460735 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a230e10-8564-4f7e-9a45-bbafbc533943 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:08.461175 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a230e10-8564-4f7e-9a45-bbafbc533943 None admin] PATH_INFO: `/v3/projects/afb9ca9466054e008e3b052866da363d` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:08.463261 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a230e10-8564-4f7e-9a45-bbafbc533943 None admin] RBAC: Authorizing `identity:get_project(project_id=afb9ca9466054e008e3b052866da363d)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:08.466790 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a230e10-8564-4f7e-9a45-bbafbc533943 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:08.472050 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 176/358] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Jul 26 13:33:08 2022] GET /identity/v3/projects/afb9ca9466054e008e3b052866da363d => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:08.472050 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3164be44-d69d-4686-a385-7ec2199060ba None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:08.472050 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3164be44-d69d-4686-a385-7ec2199060ba None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:08.472050 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3164be44-d69d-4686-a385-7ec2199060ba None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:08.475132 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 177/359] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:33:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:33:08.478243 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da71057-13ce-48fe-96b9-cf4f4502543c None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:08.478637 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da71057-13ce-48fe-96b9-cf4f4502543c None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:08.479028 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da71057-13ce-48fe-96b9-cf4f4502543c None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:08.485586 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 178/360] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:33:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:33:08.487338 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da71057-13ce-48fe-96b9-cf4f4502543c None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:08.540147 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:08.560085 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 183/361] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Jul 26 13:33:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 26 13:33:08.562233 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332774c8-dfc6-4e92-898e-1d00e6a47b40 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:08.562731 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332774c8-dfc6-4e92-898e-1d00e6a47b40 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:08.563294 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332774c8-dfc6-4e92-898e-1d00e6a47b40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:08.583025 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-332774c8-dfc6-4e92-898e-1d00e6a47b40 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:33:08.584243 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-332774c8-dfc6-4e92-898e-1d00e6a47b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:33:08.641003 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da71057-13ce-48fe-96b9-cf4f4502543c None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:08.683553 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d40f167-a40a-48da-b81c-af0abc2e4305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:08.684211 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d40f167-a40a-48da-b81c-af0abc2e4305 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:08.684423 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 184/362] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:33:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 13:33:08.684523 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d40f167-a40a-48da-b81c-af0abc2e4305 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:08.684757 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d40f167-a40a-48da-b81c-af0abc2e4305 None admin] PATH_INFO: `/v3/groups/04331d538f1c4d479d659675e703da04` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:08.686612 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d40f167-a40a-48da-b81c-af0abc2e4305 None admin] RBAC: Authorizing `identity:get_group(group_id=04331d538f1c4d479d659675e703da04)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:08.686813 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082c49ec-8e25-4894-9f37-791deafd97a9 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:08.687252 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082c49ec-8e25-4894-9f37-791deafd97a9 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:08.687690 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082c49ec-8e25-4894-9f37-791deafd97a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:08.688012 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d40f167-a40a-48da-b81c-af0abc2e4305 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:08.689815 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 179/363] 10.222.0.58 () {56 vars in 1180 bytes} [Tue Jul 26 13:33:08 2022] GET /identity/v3/groups/04331d538f1c4d479d659675e703da04 => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:08.690445 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d40f167-a40a-48da-b81c-af0abc2e4305 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:08.711572 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-082c49ec-8e25-4894-9f37-791deafd97a9 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:33:08.712904 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-082c49ec-8e25-4894-9f37-791deafd97a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:33:08.765667 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:08.807381 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 185/364] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:33:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 13:33:08.808578 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082c49ec-8e25-4894-9f37-791deafd97a9 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:08.830663 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d40f167-a40a-48da-b81c-af0abc2e4305 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:08.879880 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:08.887374 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61427509-d6df-4750-b881-f45898e54209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:08.888185 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61427509-d6df-4750-b881-f45898e54209 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:08.888476 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61427509-d6df-4750-b881-f45898e54209 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:08.888755 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61427509-d6df-4750-b881-f45898e54209 None admin] PATH_INFO: `/v3/projects/6ad88492b683452a81ab7150c22b665e` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:08.890246 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61427509-d6df-4750-b881-f45898e54209 None admin] RBAC: Authorizing `identity:get_project(project_id=6ad88492b683452a81ab7150c22b665e)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:08.892391 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61427509-d6df-4750-b881-f45898e54209 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:08.897618 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 180/365] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Jul 26 13:33:08 2022] GET /identity/v3/projects/6ad88492b683452a81ab7150c22b665e => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:08.898786 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61427509-d6df-4750-b881-f45898e54209 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:08.962389 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082c49ec-8e25-4894-9f37-791deafd97a9 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:09.009422 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20ee237-cbb3-4a85-aab6-e3153d429b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:09.010106 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20ee237-cbb3-4a85-aab6-e3153d429b03 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:09.010393 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20ee237-cbb3-4a85-aab6-e3153d429b03 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:09.010677 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20ee237-cbb3-4a85-aab6-e3153d429b03 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:09.012666 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20ee237-cbb3-4a85-aab6-e3153d429b03 None admin] RBAC: Adding query filter params (group.id=04331d538f1c4d479d659675e703da04, role.id=5cb730ec786743c3bb8af83da90f9309, scope.project.id=afb9ca9466054e008e3b052866da363d) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:09.012666 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20ee237-cbb3-4a85-aab6-e3153d429b03 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:09.014212 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20ee237-cbb3-4a85-aab6-e3153d429b03 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:09.025931 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 186/366] 10.222.0.58 () {56 vars in 1376 bytes} [Tue Jul 26 13:33:08 2022] GET /identity/v3/role_assignments?group.id=04331d538f1c4d479d659675e703da04&scope.project.id=afb9ca9466054e008e3b052866da363d&role.id=5cb730ec786743c3bb8af83da90f9309 => generated 260 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:09.027153 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20ee237-cbb3-4a85-aab6-e3153d429b03 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:09.087735 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61427509-d6df-4750-b881-f45898e54209 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:09.143507 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:09.169402 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac92b190-d706-4dd3-ac97-12a9af28641f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:09.170360 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac92b190-d706-4dd3-ac97-12a9af28641f None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:09.170869 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac92b190-d706-4dd3-ac97-12a9af28641f None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:09.171290 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac92b190-d706-4dd3-ac97-12a9af28641f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:09.198404 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac92b190-d706-4dd3-ac97-12a9af28641f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:33:09.214848 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-ac92b190-d706-4dd3-ac97-12a9af28641f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 13:33:09.214848 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:09.214848 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:09.214848 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:09.214848 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:09.214848 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:09.214848 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:09.214848 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:09.214848 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:09.214848 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:09.214848 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:09.214848 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:09.214848 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:33:09.214848 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:33:09.214848 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:33:09.214848 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:33:09.215835 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:33:09.215835 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:33:09.215835 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:33:09.215835 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:33:09.215835 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:09.215835 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:09.215835 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:09.215835 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:09.215835 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:09.215835 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:09.215835 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:09.215835 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:09.215835 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:09.215835 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:09.215835 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:09.215835 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:09.216656 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:09.216656 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:09.216656 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:09.216656 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:09.216656 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:33:09.216656 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:33:09.216656 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:33:09.216656 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:33:09.216656 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:33:09.216656 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:33:09.216656 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 13:33:09.216656 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:33:09.218729 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 181/367] 10.222.0.58 () {56 vars in 1096 bytes} [Tue Jul 26 13:33:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:09.222858 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac92b190-d706-4dd3-ac97-12a9af28641f None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:09.270791 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20ee237-cbb3-4a85-aab6-e3153d429b03 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:09.307433 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:09.365355 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7735beec-f22c-4b0c-b280-05f378415a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:09.366218 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7735beec-f22c-4b0c-b280-05f378415a52 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:09.366600 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7735beec-f22c-4b0c-b280-05f378415a52 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:09.366978 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7735beec-f22c-4b0c-b280-05f378415a52 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:09.369198 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac92b190-d706-4dd3-ac97-12a9af28641f None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:09.389638 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7735beec-f22c-4b0c-b280-05f378415a52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:33:09.407559 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-7735beec-f22c-4b0c-b280-05f378415a52 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 13:33:09.407559 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:09.407559 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:09.407559 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:09.407559 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:09.407559 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:09.407559 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:09.407559 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:09.407559 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:09.407559 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:09.407559 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:09.407559 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:09.407559 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:33:09.407559 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:33:09.407559 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:33:09.407559 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:33:09.408533 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:33:09.408533 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:33:09.408533 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:33:09.408533 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:33:09.408533 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:09.408533 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:09.408533 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:09.408533 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:09.408533 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:09.408533 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:09.408533 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:09.408533 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:09.408533 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:09.408533 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:09.408533 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:09.408533 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:09.409343 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:09.409343 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:09.409343 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:09.409343 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:09.409343 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:33:09.409343 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:33:09.409343 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:33:09.409343 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:33:09.409343 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:33:09.409343 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:33:09.409343 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 13:33:09.409343 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:33:09.409343 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 187/368] 10.222.0.58 () {56 vars in 1114 bytes} [Tue Jul 26 13:33:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 382 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:09.412786 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c918d647-0375-4427-8b01-86fef964982c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:09.413726 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c918d647-0375-4427-8b01-86fef964982c None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:09.414201 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c918d647-0375-4427-8b01-86fef964982c None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:09.414638 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c918d647-0375-4427-8b01-86fef964982c None admin] PATH_INFO: `/v3/groups/04331d538f1c4d479d659675e703da04` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:09.414951 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7735beec-f22c-4b0c-b280-05f378415a52 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:09.416369 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c918d647-0375-4427-8b01-86fef964982c None admin] RBAC: Authorizing `identity:get_group(group_id=04331d538f1c4d479d659675e703da04)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:09.418713 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c918d647-0375-4427-8b01-86fef964982c None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:09.420594 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 182/369] 10.222.0.58 () {56 vars in 1180 bytes} [Tue Jul 26 13:33:09 2022] GET /identity/v3/groups/04331d538f1c4d479d659675e703da04 => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:09.423145 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c918d647-0375-4427-8b01-86fef964982c None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:09.494380 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:09.553504 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c918d647-0375-4427-8b01-86fef964982c None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:09.603029 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2309d6-c5ac-427c-8fa3-41b483b2b3b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:09.604129 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2309d6-c5ac-427c-8fa3-41b483b2b3b3 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:09.605233 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2309d6-c5ac-427c-8fa3-41b483b2b3b3 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:09.605761 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2309d6-c5ac-427c-8fa3-41b483b2b3b3 None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:09.606814 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2309d6-c5ac-427c-8fa3-41b483b2b3b3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:09.607368 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2309d6-c5ac-427c-8fa3-41b483b2b3b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:09.609144 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2309d6-c5ac-427c-8fa3-41b483b2b3b3 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:09.624976 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 183/370] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Jul 26 13:33:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:09.627029 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2309d6-c5ac-427c-8fa3-41b483b2b3b3 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:09.671885 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7735beec-f22c-4b0c-b280-05f378415a52 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:09.715141 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70424439-8d75-4415-a6d4-280f0d82af19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:09.715791 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70424439-8d75-4415-a6d4-280f0d82af19 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:09.716235 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70424439-8d75-4415-a6d4-280f0d82af19 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:09.716565 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70424439-8d75-4415-a6d4-280f0d82af19 None admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:09.717302 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70424439-8d75-4415-a6d4-280f0d82af19 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:09.717731 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70424439-8d75-4415-a6d4-280f0d82af19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:09.719169 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70424439-8d75-4415-a6d4-280f0d82af19 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:09.737411 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 188/371] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Jul 26 13:33:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:09.738489 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70424439-8d75-4415-a6d4-280f0d82af19 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:09.750565 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2309d6-c5ac-427c-8fa3-41b483b2b3b3 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:09.767381 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:09.781430 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:09.792093 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7173cfdb-1d83-4442-aac7-59a8ef4905fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:09.793613 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7173cfdb-1d83-4442-aac7-59a8ef4905fd None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:09.793817 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7173cfdb-1d83-4442-aac7-59a8ef4905fd None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:09.794192 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7173cfdb-1d83-4442-aac7-59a8ef4905fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:09.795122 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7173cfdb-1d83-4442-aac7-59a8ef4905fd None admin] RBAC: Adding query filter params (group.id=04331d538f1c4d479d659675e703da04, role.id=759f389b221049c4b01faf3b984cd8d7, scope.project.id=6ad88492b683452a81ab7150c22b665e) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:09.795895 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7173cfdb-1d83-4442-aac7-59a8ef4905fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:09.797102 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7173cfdb-1d83-4442-aac7-59a8ef4905fd None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:09.804171 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 184/372] 10.222.0.58 () {56 vars in 1376 bytes} [Tue Jul 26 13:33:09 2022] GET /identity/v3/role_assignments?group.id=04331d538f1c4d479d659675e703da04&scope.project.id=6ad88492b683452a81ab7150c22b665e&role.id=759f389b221049c4b01faf3b984cd8d7 => generated 260 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:09.804933 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7173cfdb-1d83-4442-aac7-59a8ef4905fd None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:09.865358 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70424439-8d75-4415-a6d4-280f0d82af19 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:09.908443 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50221c28-ae64-437d-b3bc-53d809371f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:09.909540 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50221c28-ae64-437d-b3bc-53d809371f07 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:09.910033 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50221c28-ae64-437d-b3bc-53d809371f07 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:09.910527 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50221c28-ae64-437d-b3bc-53d809371f07 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:09.923421 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7173cfdb-1d83-4442-aac7-59a8ef4905fd None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:09.964684 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda44d5a-5fda-4436-84ae-82b0beea0a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:09.965194 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda44d5a-5fda-4436-84ae-82b0beea0a6e None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:09.965527 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda44d5a-5fda-4436-84ae-82b0beea0a6e None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:09.965786 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda44d5a-5fda-4436-84ae-82b0beea0a6e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:10.001240 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50221c28-ae64-437d-b3bc-53d809371f07 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:10.003217 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50221c28-ae64-437d-b3bc-53d809371f07 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:10.019234 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:10.046308 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda44d5a-5fda-4436-84ae-82b0beea0a6e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:10.047158 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:10.047875 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda44d5a-5fda-4436-84ae-82b0beea0a6e None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:10.131797 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 189/373] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Jul 26 13:33:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 13:33:10.134590 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af369398-6a81-4495-964a-d0e06786c22c None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:10.135130 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af369398-6a81-4495-964a-d0e06786c22c None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:10.135615 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af369398-6a81-4495-964a-d0e06786c22c None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:10.138035 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 190/374] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:33:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:33:10.138996 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af369398-6a81-4495-964a-d0e06786c22c None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:10.185693 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 185/375] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Jul 26 13:33:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 13:33:10.185693 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40b419e-f288-4710-9a3d-2a1a01e54dca None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:10.185693 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40b419e-f288-4710-9a3d-2a1a01e54dca None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:10.185693 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40b419e-f288-4710-9a3d-2a1a01e54dca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:10.201765 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-d40b419e-f288-4710-9a3d-2a1a01e54dca None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:33:10.202974 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-d40b419e-f288-4710-9a3d-2a1a01e54dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:33:10.271828 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af369398-6a81-4495-964a-d0e06786c22c None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:10.279164 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 186/376] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:33:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 13:33:10.280243 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40b419e-f288-4710-9a3d-2a1a01e54dca None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:10.291101 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:10.321050 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb74bc46-ddb0-4628-8346-5d174abd717e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:10.322119 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb74bc46-ddb0-4628-8346-5d174abd717e None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:10.322619 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb74bc46-ddb0-4628-8346-5d174abd717e None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:10.323089 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb74bc46-ddb0-4628-8346-5d174abd717e None admin] PATH_INFO: `/v3/projects/6ad88492b683452a81ab7150c22b665e` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:10.325287 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb74bc46-ddb0-4628-8346-5d174abd717e None admin] RBAC: Authorizing `identity:get_project(project_id=6ad88492b683452a81ab7150c22b665e)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:10.327045 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb74bc46-ddb0-4628-8346-5d174abd717e None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:10.329323 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 191/377] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Jul 26 13:33:10 2022] GET /identity/v3/projects/6ad88492b683452a81ab7150c22b665e => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:10.330353 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb74bc46-ddb0-4628-8346-5d174abd717e None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:10.335470 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:10.399398 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40b419e-f288-4710-9a3d-2a1a01e54dca None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:10.437960 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac81fe7-2572-4210-9f67-e67e7444cde4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:10.438536 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac81fe7-2572-4210-9f67-e67e7444cde4 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:10.438822 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac81fe7-2572-4210-9f67-e67e7444cde4 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:10.439215 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac81fe7-2572-4210-9f67-e67e7444cde4 None admin] PATH_INFO: `/v3/projects/2e341f5b1be34081bfdda39547d48a7c` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:10.440482 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac81fe7-2572-4210-9f67-e67e7444cde4 None admin] RBAC: Authorizing `identity:get_project(project_id=2e341f5b1be34081bfdda39547d48a7c)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:10.442132 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac81fe7-2572-4210-9f67-e67e7444cde4 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:10.444064 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 187/378] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Jul 26 13:33:10 2022] GET /identity/v3/projects/2e341f5b1be34081bfdda39547d48a7c => generated 344 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:10.444688 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac81fe7-2572-4210-9f67-e67e7444cde4 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:10.459367 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb74bc46-ddb0-4628-8346-5d174abd717e None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:10.499720 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786baced-8d77-4cb0-8089-48f7477dded6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:10.500438 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786baced-8d77-4cb0-8089-48f7477dded6 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:10.500720 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786baced-8d77-4cb0-8089-48f7477dded6 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:10.501046 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786baced-8d77-4cb0-8089-48f7477dded6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:10.514879 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786baced-8d77-4cb0-8089-48f7477dded6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:33:10.521652 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-786baced-8d77-4cb0-8089-48f7477dded6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 13:33:10.521652 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:10.521652 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:10.521652 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:10.521652 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:10.521652 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:10.521652 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:10.521652 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:10.521652 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:10.521652 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:10.521652 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:10.521652 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:10.521652 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:33:10.521652 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:33:10.521652 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:33:10.521652 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:33:10.522625 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:33:10.522625 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:33:10.522625 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:33:10.522625 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:33:10.522625 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:10.522625 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:10.522625 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:10.522625 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:10.522625 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:10.522625 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:10.522625 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:10.522625 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:10.522625 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:10.522625 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:10.522625 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:10.522625 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:10.523491 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:10.523491 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:10.523491 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:10.523491 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:10.523491 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:33:10.523491 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:33:10.523491 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:33:10.523491 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:33:10.523491 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:33:10.523491 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:33:10.523491 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 13:33:10.523491 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:33:10.523491 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 192/379] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Jul 26 13:33:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:10.524676 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786baced-8d77-4cb0-8089-48f7477dded6 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:10.533332 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:10.559234 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac81fe7-2572-4210-9f67-e67e7444cde4 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:10.598314 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909709c9-c726-43e1-b05e-6965b7cf9c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:10.598988 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909709c9-c726-43e1-b05e-6965b7cf9c80 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:10.599273 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909709c9-c726-43e1-b05e-6965b7cf9c80 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:10.599555 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909709c9-c726-43e1-b05e-6965b7cf9c80 None admin] PATH_INFO: `/v3/groups/04331d538f1c4d479d659675e703da04` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:10.600886 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909709c9-c726-43e1-b05e-6965b7cf9c80 None admin] RBAC: Authorizing `identity:get_group(group_id=04331d538f1c4d479d659675e703da04)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:10.602370 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:10.602521 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909709c9-c726-43e1-b05e-6965b7cf9c80 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:10.604531 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 188/380] 10.222.0.58 () {56 vars in 1180 bytes} [Tue Jul 26 13:33:10 2022] GET /identity/v3/groups/04331d538f1c4d479d659675e703da04 => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:10.605467 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909709c9-c726-43e1-b05e-6965b7cf9c80 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:10.653869 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786baced-8d77-4cb0-8089-48f7477dded6 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:10.694916 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cd0121-b3d6-4a49-b297-d25f82a9381b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:10.696419 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cd0121-b3d6-4a49-b297-d25f82a9381b None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:10.696962 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cd0121-b3d6-4a49-b297-d25f82a9381b None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:10.697426 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cd0121-b3d6-4a49-b297-d25f82a9381b None admin] PATH_INFO: `/v3/groups/90ce4124fd5f4779a13930568b5e0aae` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:10.704069 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cd0121-b3d6-4a49-b297-d25f82a9381b None admin] RBAC: Authorizing `identity:get_group(group_id=90ce4124fd5f4779a13930568b5e0aae)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:10.705973 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cd0121-b3d6-4a49-b297-d25f82a9381b None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:10.708429 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 193/381] 10.222.0.58 () {56 vars in 1180 bytes} [Tue Jul 26 13:33:10 2022] GET /identity/v3/groups/90ce4124fd5f4779a13930568b5e0aae => generated 229 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:10.709416 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cd0121-b3d6-4a49-b297-d25f82a9381b None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:10.717714 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909709c9-c726-43e1-b05e-6965b7cf9c80 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:10.755944 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4f4bef-dbb8-4593-aec3-c675bbdf6b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:10.756579 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4f4bef-dbb8-4593-aec3-c675bbdf6b39 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:10.756848 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4f4bef-dbb8-4593-aec3-c675bbdf6b39 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:10.757124 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4f4bef-dbb8-4593-aec3-c675bbdf6b39 None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:10.757891 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4f4bef-dbb8-4593-aec3-c675bbdf6b39 None admin] RBAC: Adding query filter params (name=member) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:10.758216 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4f4bef-dbb8-4593-aec3-c675bbdf6b39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:10.759770 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4f4bef-dbb8-4593-aec3-c675bbdf6b39 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:10.772603 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 189/382] 10.222.0.58 () {56 vars in 1101 bytes} [Tue Jul 26 13:33:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:10.773467 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4f4bef-dbb8-4593-aec3-c675bbdf6b39 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:10.792461 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:10.850517 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cd0121-b3d6-4a49-b297-d25f82a9381b None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:10.886325 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:10.908615 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553748c5-8a02-4a37-b60c-b61114e9f786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:10.909253 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553748c5-8a02-4a37-b60c-b61114e9f786 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:10.909562 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553748c5-8a02-4a37-b60c-b61114e9f786 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:10.909838 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553748c5-8a02-4a37-b60c-b61114e9f786 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:10.910640 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553748c5-8a02-4a37-b60c-b61114e9f786 None admin] RBAC: Adding query filter params (group.id=04331d538f1c4d479d659675e703da04, role.id=5cb730ec786743c3bb8af83da90f9309, scope.project.id=6ad88492b683452a81ab7150c22b665e) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:10.910951 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553748c5-8a02-4a37-b60c-b61114e9f786 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:10.912396 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553748c5-8a02-4a37-b60c-b61114e9f786 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:10.919280 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 194/383] 10.222.0.58 () {56 vars in 1376 bytes} [Tue Jul 26 13:33:10 2022] GET /identity/v3/role_assignments?group.id=04331d538f1c4d479d659675e703da04&scope.project.id=6ad88492b683452a81ab7150c22b665e&role.id=5cb730ec786743c3bb8af83da90f9309 => generated 260 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:10.919951 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553748c5-8a02-4a37-b60c-b61114e9f786 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:10.925794 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4f4bef-dbb8-4593-aec3-c675bbdf6b39 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:10.974280 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6159bca-e103-4492-b7d0-c8613809ec8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:10.974847 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6159bca-e103-4492-b7d0-c8613809ec8e None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:10.975143 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6159bca-e103-4492-b7d0-c8613809ec8e None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:10.975432 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6159bca-e103-4492-b7d0-c8613809ec8e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:10.976803 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6159bca-e103-4492-b7d0-c8613809ec8e None admin] RBAC: Adding query filter params (group.id=90ce4124fd5f4779a13930568b5e0aae, role.id=6e0201a9a6484f7d8b112b1eafff70bd, scope.project.id=2e341f5b1be34081bfdda39547d48a7c) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:10.977135 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6159bca-e103-4492-b7d0-c8613809ec8e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:10.978854 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6159bca-e103-4492-b7d0-c8613809ec8e None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:10.987173 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 190/384] 10.222.0.58 () {56 vars in 1376 bytes} [Tue Jul 26 13:33:10 2022] GET /identity/v3/role_assignments?group.id=90ce4124fd5f4779a13930568b5e0aae&scope.project.id=2e341f5b1be34081bfdda39547d48a7c&role.id=6e0201a9a6484f7d8b112b1eafff70bd => generated 260 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:11.136254 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553748c5-8a02-4a37-b60c-b61114e9f786 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:11.185564 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5301f0db-2b6c-47b8-b8f6-1914e7d1013c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:11.193061 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5301f0db-2b6c-47b8-b8f6-1914e7d1013c None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:11.193346 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5301f0db-2b6c-47b8-b8f6-1914e7d1013c None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:11.193671 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5301f0db-2b6c-47b8-b8f6-1914e7d1013c None admin] PATH_INFO: `/v3/groups/04331d538f1c4d479d659675e703da04` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:11.195955 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5301f0db-2b6c-47b8-b8f6-1914e7d1013c None admin] RBAC: Authorizing `identity:get_group(group_id=04331d538f1c4d479d659675e703da04)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:11.197439 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5301f0db-2b6c-47b8-b8f6-1914e7d1013c None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:11.199568 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 195/385] 10.222.0.58 () {56 vars in 1180 bytes} [Tue Jul 26 13:33:10 2022] GET /identity/v3/groups/04331d538f1c4d479d659675e703da04 => generated 226 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:11.205585 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6159bca-e103-4492-b7d0-c8613809ec8e None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:11.304188 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:11.368903 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6159bca-e103-4492-b7d0-c8613809ec8e None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:11.436184 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f912b5f-8fbd-4429-b4b0-d5e65d67e957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:11.437148 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f912b5f-8fbd-4429-b4b0-d5e65d67e957 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:11.437607 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f912b5f-8fbd-4429-b4b0-d5e65d67e957 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:11.438038 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f912b5f-8fbd-4429-b4b0-d5e65d67e957 None admin] PATH_INFO: `/v3/projects/afb9ca9466054e008e3b052866da363d` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:11.442523 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f912b5f-8fbd-4429-b4b0-d5e65d67e957 None admin] RBAC: Authorizing `identity:get_project(project_id=afb9ca9466054e008e3b052866da363d)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:11.445089 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f912b5f-8fbd-4429-b4b0-d5e65d67e957 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:11.447978 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 191/386] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Jul 26 13:33:11 2022] GET /identity/v3/projects/afb9ca9466054e008e3b052866da363d => generated 298 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:11.453516 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5301f0db-2b6c-47b8-b8f6-1914e7d1013c None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:11.556262 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:11.616217 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5301f0db-2b6c-47b8-b8f6-1914e7d1013c None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:11.666793 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d64da9a-a907-4069-a7d5-49272521d8af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:11.667658 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d64da9a-a907-4069-a7d5-49272521d8af None admin] REQUEST_METHOD: `PUT` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:11.667949 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d64da9a-a907-4069-a7d5-49272521d8af None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:11.668230 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d64da9a-a907-4069-a7d5-49272521d8af None admin] PATH_INFO: `/v3/projects/afb9ca9466054e008e3b052866da363d/groups/04331d538f1c4d479d659675e703da04/roles/759f389b221049c4b01faf3b984cd8d7` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:11.670321 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d64da9a-a907-4069-a7d5-49272521d8af None admin] RBAC: Authorizing `identity:create_grant(project_id=afb9ca9466054e008e3b052866da363d, group_id=04331d538f1c4d479d659675e703da04, role_id=759f389b221049c4b01faf3b984cd8d7)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:11.671789 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d64da9a-a907-4069-a7d5-49272521d8af None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:11.681866 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aad9d21-fb3d-4a48-b669-8ee7c75073ce None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:11.682159 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aad9d21-fb3d-4a48-b669-8ee7c75073ce None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:11.682434 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aad9d21-fb3d-4a48-b669-8ee7c75073ce None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:11.684147 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 192/387] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:33:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:33:11.690163 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36af8263-539a-4dae-b3a1-6cfe71563ffe None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:11.690469 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36af8263-539a-4dae-b3a1-6cfe71563ffe None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:11.690724 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36af8263-539a-4dae-b3a1-6cfe71563ffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:11.713087 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 196/388] 10.222.0.58 () {58 vars in 1442 bytes} [Tue Jul 26 13:33:11 2022] PUT /identity/v3/projects/afb9ca9466054e008e3b052866da363d/groups/04331d538f1c4d479d659675e703da04/roles/759f389b221049c4b01faf3b984cd8d7 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:33:11.718406 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-36af8263-539a-4dae-b3a1-6cfe71563ffe None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:33:11.719670 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-36af8263-539a-4dae-b3a1-6cfe71563ffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:33:11.802656 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 193/389] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:33:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 13:33:11.843168 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d64da9a-a907-4069-a7d5-49272521d8af None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:11.972519 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d64da9a-a907-4069-a7d5-49272521d8af None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:11.996095 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:12.016644 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb297a2-5c1b-4326-9442-0ead9b1592db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:12.017314 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb297a2-5c1b-4326-9442-0ead9b1592db None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:12.017651 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb297a2-5c1b-4326-9442-0ead9b1592db None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:12.017930 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb297a2-5c1b-4326-9442-0ead9b1592db None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:12.046331 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb297a2-5c1b-4326-9442-0ead9b1592db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:33:12.052175 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-5cb297a2-5c1b-4326-9442-0ead9b1592db None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 13:33:12.052175 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:12.052175 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:12.052175 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:12.052175 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:12.052175 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:12.052175 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:12.052175 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:12.052175 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:12.052175 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:12.052175 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:12.052175 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:12.052175 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:33:12.052175 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:33:12.052175 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:33:12.052175 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:33:12.053043 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:33:12.053043 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:33:12.053043 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:33:12.053043 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:33:12.053043 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:12.053043 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:12.053043 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:12.053043 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:12.053043 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:12.053043 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:12.053043 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:12.053043 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:12.053043 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:12.053043 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:12.053043 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:12.053043 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:12.053867 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:12.053867 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:12.053867 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:12.053867 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:12.053867 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:33:12.053867 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:33:12.053867 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:33:12.053867 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:33:12.053867 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:33:12.053867 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:33:12.053867 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 13:33:12.053867 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:33:12.055345 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 197/390] 10.222.0.58 () {56 vars in 1114 bytes} [Tue Jul 26 13:33:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:12.060414 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36af8263-539a-4dae-b3a1-6cfe71563ffe None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:12.155062 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:12.192986 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36af8263-539a-4dae-b3a1-6cfe71563ffe None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:12.239538 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aa0650-fe86-4e56-865e-60a4c2c4414e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:12.240180 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aa0650-fe86-4e56-865e-60a4c2c4414e None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:12.240468 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aa0650-fe86-4e56-865e-60a4c2c4414e None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:12.240745 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aa0650-fe86-4e56-865e-60a4c2c4414e None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:12.241509 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5aa0650-fe86-4e56-865e-60a4c2c4414e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:12.241815 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5aa0650-fe86-4e56-865e-60a4c2c4414e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:12.243193 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5aa0650-fe86-4e56-865e-60a4c2c4414e None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:12.256845 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 194/391] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Jul 26 13:33:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:12.263169 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb297a2-5c1b-4326-9442-0ead9b1592db None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:12.442733 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb297a2-5c1b-4326-9442-0ead9b1592db None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:12.495267 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f219160-f392-4e25-aeee-56ae53e36083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:12.495958 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f219160-f392-4e25-aeee-56ae53e36083 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:12.496244 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f219160-f392-4e25-aeee-56ae53e36083 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:12.496523 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f219160-f392-4e25-aeee-56ae53e36083 None admin] PATH_INFO: `/v3/groups/04331d538f1c4d479d659675e703da04` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:12.497865 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f219160-f392-4e25-aeee-56ae53e36083 None admin] RBAC: Authorizing `identity:get_group(group_id=04331d538f1c4d479d659675e703da04)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:12.499345 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f219160-f392-4e25-aeee-56ae53e36083 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:12.501379 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 198/392] 10.222.0.58 () {56 vars in 1180 bytes} [Tue Jul 26 13:33:12 2022] GET /identity/v3/groups/04331d538f1c4d479d659675e703da04 => generated 226 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:12.507234 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aa0650-fe86-4e56-865e-60a4c2c4414e None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:12.663897 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aa0650-fe86-4e56-865e-60a4c2c4414e None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:12.700674 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:12.706591 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0938b88-6496-4951-be09-656ef55c3a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:12.707300 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0938b88-6496-4951-be09-656ef55c3a6f None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:12.707892 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0938b88-6496-4951-be09-656ef55c3a6f None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:12.708167 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0938b88-6496-4951-be09-656ef55c3a6f None admin] PATH_INFO: `/v3/projects/afb9ca9466054e008e3b052866da363d` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:12.709411 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0938b88-6496-4951-be09-656ef55c3a6f None admin] RBAC: Authorizing `identity:get_project(project_id=afb9ca9466054e008e3b052866da363d)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:12.710881 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0938b88-6496-4951-be09-656ef55c3a6f None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:12.713122 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 195/393] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Jul 26 13:33:12 2022] GET /identity/v3/projects/afb9ca9466054e008e3b052866da363d => generated 298 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:12.718523 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f219160-f392-4e25-aeee-56ae53e36083 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:12.790756 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:12.854487 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f219160-f392-4e25-aeee-56ae53e36083 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:12.895837 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4832e718-bee0-4842-8012-dffbef3675ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:12.896918 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4832e718-bee0-4842-8012-dffbef3675ec None admin] REQUEST_METHOD: `PUT` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:12.897411 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4832e718-bee0-4842-8012-dffbef3675ec None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:12.897918 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4832e718-bee0-4842-8012-dffbef3675ec None admin] PATH_INFO: `/v3/projects/afb9ca9466054e008e3b052866da363d/groups/04331d538f1c4d479d659675e703da04/roles/5cb730ec786743c3bb8af83da90f9309` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:12.900399 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4832e718-bee0-4842-8012-dffbef3675ec None admin] RBAC: Authorizing `identity:create_grant(project_id=afb9ca9466054e008e3b052866da363d, group_id=04331d538f1c4d479d659675e703da04, role_id=5cb730ec786743c3bb8af83da90f9309)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:12.902156 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4832e718-bee0-4842-8012-dffbef3675ec None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:12.926337 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 199/394] 10.222.0.58 () {58 vars in 1442 bytes} [Tue Jul 26 13:33:12 2022] PUT /identity/v3/projects/afb9ca9466054e008e3b052866da363d/groups/04331d538f1c4d479d659675e703da04/roles/5cb730ec786743c3bb8af83da90f9309 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:33:12.932102 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67b26b8-beba-4623-b0ee-f588fb593cd3 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:12.933902 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67b26b8-beba-4623-b0ee-f588fb593cd3 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:12.934187 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67b26b8-beba-4623-b0ee-f588fb593cd3 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:12.935755 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 196/395] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:33:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:33:12.945070 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95e66ff-8ad4-4476-8ab7-29baf144ea2c None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:12.945432 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95e66ff-8ad4-4476-8ab7-29baf144ea2c None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:12.945734 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95e66ff-8ad4-4476-8ab7-29baf144ea2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:12.971556 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-f95e66ff-8ad4-4476-8ab7-29baf144ea2c None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:33:12.973387 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-f95e66ff-8ad4-4476-8ab7-29baf144ea2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:33:13.059297 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 200/396] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:33:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 13:33:13.081136 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67b26b8-beba-4623-b0ee-f588fb593cd3 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:13.217865 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67b26b8-beba-4623-b0ee-f588fb593cd3 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:13.263626 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fb9862-8572-4bc4-a000-d579e62046de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:13.264589 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fb9862-8572-4bc4-a000-d579e62046de None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:13.265056 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fb9862-8572-4bc4-a000-d579e62046de None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:13.265535 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fb9862-8572-4bc4-a000-d579e62046de None admin] PATH_INFO: `/v3/roles/member` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:13.280467 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fb9862-8572-4bc4-a000-d579e62046de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:33:13.287987 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-34fb9862-8572-4bc4-a000-d579e62046de None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 13:33:13.287987 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:13.287987 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:13.287987 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:13.287987 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:13.287987 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:13.287987 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:13.287987 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:13.287987 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:13.287987 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:13.287987 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:13.287987 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:13.287987 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:33:13.287987 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:33:13.287987 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:33:13.287987 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:33:13.288939 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:33:13.288939 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:33:13.288939 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:33:13.288939 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:33:13.288939 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:13.288939 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:13.288939 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:13.288939 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:13.288939 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:13.288939 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:13.288939 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:13.288939 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:13.288939 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:13.288939 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:13.288939 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:13.288939 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:13.289776 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:13.289776 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:13.289776 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:13.289776 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:13.289776 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:33:13.289776 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:33:13.289776 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:33:13.289776 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:33:13.289776 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:33:13.289776 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:33:13.289776 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 13:33:13.289776 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:33:13.292725 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 197/397] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Jul 26 13:33:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:13.299945 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95e66ff-8ad4-4476-8ab7-29baf144ea2c None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:13.344261 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:13.414631 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95e66ff-8ad4-4476-8ab7-29baf144ea2c None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:13.451807 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6510c807-fb69-4719-82b6-78536648a7b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:13.452706 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6510c807-fb69-4719-82b6-78536648a7b4 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:13.453143 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6510c807-fb69-4719-82b6-78536648a7b4 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:13.453600 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6510c807-fb69-4719-82b6-78536648a7b4 None admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:13.454526 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6510c807-fb69-4719-82b6-78536648a7b4 None admin] RBAC: Adding query filter params (name=member) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:13.454997 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6510c807-fb69-4719-82b6-78536648a7b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:13.456581 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6510c807-fb69-4719-82b6-78536648a7b4 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:13.466948 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 201/398] 10.222.0.58 () {56 vars in 1101 bytes} [Tue Jul 26 13:33:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:13.472763 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fb9862-8572-4bc4-a000-d579e62046de None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:13.544470 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:13.597759 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fb9862-8572-4bc4-a000-d579e62046de None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:13.639033 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60699cb-4ac3-4b24-98e7-5a065ea6d654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:13.639994 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60699cb-4ac3-4b24-98e7-5a065ea6d654 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:13.640491 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60699cb-4ac3-4b24-98e7-5a065ea6d654 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:13.640914 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60699cb-4ac3-4b24-98e7-5a065ea6d654 None admin] PATH_INFO: `/v3/groups/04331d538f1c4d479d659675e703da04` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:13.642497 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60699cb-4ac3-4b24-98e7-5a065ea6d654 None admin] RBAC: Authorizing `identity:get_group(group_id=04331d538f1c4d479d659675e703da04)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:13.644138 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60699cb-4ac3-4b24-98e7-5a065ea6d654 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:13.646325 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 198/399] 10.222.0.58 () {56 vars in 1180 bytes} [Tue Jul 26 13:33:13 2022] GET /identity/v3/groups/04331d538f1c4d479d659675e703da04 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:13.652625 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6510c807-fb69-4719-82b6-78536648a7b4 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:13.762173 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:13.765164 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6510c807-fb69-4719-82b6-78536648a7b4 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:13.803964 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e1d955-f31d-4cf6-87cb-ae10e59c9637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:13.804540 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e1d955-f31d-4cf6-87cb-ae10e59c9637 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:13.804820 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e1d955-f31d-4cf6-87cb-ae10e59c9637 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:13.805096 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e1d955-f31d-4cf6-87cb-ae10e59c9637 None admin] PATH_INFO: `/v3/projects/6ad88492b683452a81ab7150c22b665e` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:13.806313 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e1d955-f31d-4cf6-87cb-ae10e59c9637 None admin] RBAC: Authorizing `identity:get_project(project_id=6ad88492b683452a81ab7150c22b665e)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:13.807880 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e1d955-f31d-4cf6-87cb-ae10e59c9637 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:13.809967 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 202/400] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Jul 26 13:33:13 2022] GET /identity/v3/projects/6ad88492b683452a81ab7150c22b665e => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:13.815739 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60699cb-4ac3-4b24-98e7-5a065ea6d654 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:13.926684 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be6326c-3438-4b80-83b4-bb4b6b9505ca None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:13.927179 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be6326c-3438-4b80-83b4-bb4b6b9505ca None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:13.927573 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be6326c-3438-4b80-83b4-bb4b6b9505ca None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:13.929175 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 203/401] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:33:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:33:13.930397 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13945af-7dd0-44f8-a237-fa222b838e24 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:13.930684 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13945af-7dd0-44f8-a237-fa222b838e24 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:13.930961 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13945af-7dd0-44f8-a237-fa222b838e24 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:13.932506 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 204/402] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:33:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:33:13.938324 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1c366b-c014-4123-abe3-8457110640df None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:13.938629 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1c366b-c014-4123-abe3-8457110640df None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:13.938885 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1c366b-c014-4123-abe3-8457110640df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:13.965181 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-1e1c366b-c014-4123-abe3-8457110640df None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:33:13.966936 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-1e1c366b-c014-4123-abe3-8457110640df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:33:13.969586 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60699cb-4ac3-4b24-98e7-5a065ea6d654 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:13.996508 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:14.007577 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444720e3-9ef5-49cc-86d0-3517382ddc6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:14.009491 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444720e3-9ef5-49cc-86d0-3517382ddc6f None admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:14.010014 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444720e3-9ef5-49cc-86d0-3517382ddc6f None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:14.010486 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444720e3-9ef5-49cc-86d0-3517382ddc6f None admin] PATH_INFO: `/v3/projects/6ad88492b683452a81ab7150c22b665e/groups/04331d538f1c4d479d659675e703da04/roles/759f389b221049c4b01faf3b984cd8d7` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:14.013187 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444720e3-9ef5-49cc-86d0-3517382ddc6f None admin] RBAC: Authorizing `identity:create_grant(project_id=6ad88492b683452a81ab7150c22b665e, group_id=04331d538f1c4d479d659675e703da04, role_id=759f389b221049c4b01faf3b984cd8d7)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:14.015466 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444720e3-9ef5-49cc-86d0-3517382ddc6f None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:14.048542 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 199/403] 10.222.0.58 () {58 vars in 1442 bytes} [Tue Jul 26 13:33:13 2022] PUT /identity/v3/projects/6ad88492b683452a81ab7150c22b665e/groups/04331d538f1c4d479d659675e703da04/roles/759f389b221049c4b01faf3b984cd8d7 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:33:14.048542 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dd33f0-50dc-48ae-9d83-2c68b5b14574 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:14.048542 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dd33f0-50dc-48ae-9d83-2c68b5b14574 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:14.048542 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dd33f0-50dc-48ae-9d83-2c68b5b14574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:14.070098 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 205/404] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:33:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 13:33:14.088134 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-d2dd33f0-50dc-48ae-9d83-2c68b5b14574 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:33:14.090021 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-d2dd33f0-50dc-48ae-9d83-2c68b5b14574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:33:14.105892 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1c366b-c014-4123-abe3-8457110640df None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:14.238557 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 200/405] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:33:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 13:33:14.265029 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dd33f0-50dc-48ae-9d83-2c68b5b14574 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:14.276012 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:14.280601 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1c366b-c014-4123-abe3-8457110640df None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:14.317631 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4413fb-4032-4540-b621-e0471e65a393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:14.318663 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4413fb-4032-4540-b621-e0471e65a393 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:14.319139 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4413fb-4032-4540-b621-e0471e65a393 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:14.319605 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4413fb-4032-4540-b621-e0471e65a393 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:14.332447 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4413fb-4032-4540-b621-e0471e65a393 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:33:14.337968 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-fb4413fb-4032-4540-b621-e0471e65a393 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 13:33:14.337968 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:14.337968 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:14.337968 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:14.337968 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:14.337968 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:14.337968 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:14.337968 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:14.337968 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:14.337968 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:14.337968 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:14.337968 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:14.337968 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:33:14.337968 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:33:14.337968 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:33:14.337968 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:33:14.338925 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:33:14.338925 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:33:14.338925 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:33:14.338925 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:33:14.338925 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:14.338925 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:14.338925 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:14.338925 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:14.338925 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:14.338925 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:14.338925 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:14.338925 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:14.338925 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:14.338925 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:14.338925 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:14.338925 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:14.339737 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:14.339737 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:14.339737 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:14.339737 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:14.339737 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:33:14.339737 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:33:14.339737 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:33:14.339737 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:33:14.339737 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:33:14.339737 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:33:14.339737 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 13:33:14.339737 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:33:14.342421 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 206/406] 10.222.0.58 () {56 vars in 1114 bytes} [Tue Jul 26 13:33:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:14.348726 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4413fb-4032-4540-b621-e0471e65a393 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:14.388379 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dd33f0-50dc-48ae-9d83-2c68b5b14574 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:14.407797 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:14.455333 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a687a4d-3caf-4354-bdde-3333d77c5ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:14.456003 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a687a4d-3caf-4354-bdde-3333d77c5ae8 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:14.456282 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a687a4d-3caf-4354-bdde-3333d77c5ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:14.456555 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a687a4d-3caf-4354-bdde-3333d77c5ae8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:14.469159 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4413fb-4032-4540-b621-e0471e65a393 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:14.475669 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a687a4d-3caf-4354-bdde-3333d77c5ae8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:33:14.482373 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-7a687a4d-3caf-4354-bdde-3333d77c5ae8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 13:33:14.482373 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:14.482373 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:14.482373 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:14.482373 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:14.482373 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:14.482373 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:14.482373 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:14.482373 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:14.482373 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:14.482373 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:14.482373 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:14.482373 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:33:14.482373 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:33:14.482373 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:33:14.482373 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:33:14.483334 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:33:14.483334 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:33:14.483334 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:33:14.483334 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:33:14.483334 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:14.483334 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:14.483334 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:14.483334 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:14.483334 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:14.483334 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:14.483334 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:14.483334 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:14.483334 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:14.483334 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:14.483334 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:14.483334 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:14.484123 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:14.484123 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:14.484123 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:14.484123 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:14.484123 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:33:14.484123 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:33:14.484123 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:33:14.484123 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:33:14.484123 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:33:14.484123 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:33:14.484123 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 13:33:14.484123 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:33:14.484123 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 201/407] 10.222.0.58 () {56 vars in 1096 bytes} [Tue Jul 26 13:33:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 13:33:14.489740 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a687a4d-3caf-4354-bdde-3333d77c5ae8 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:14.514016 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4ed937-4ea2-45c9-abaa-31758358b7ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:14.515056 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4ed937-4ea2-45c9-abaa-31758358b7ea None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:14.515597 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4ed937-4ea2-45c9-abaa-31758358b7ea None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:14.516191 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4ed937-4ea2-45c9-abaa-31758358b7ea None admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:14.517122 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4ed937-4ea2-45c9-abaa-31758358b7ea None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:14.517661 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4ed937-4ea2-45c9-abaa-31758358b7ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:14.519321 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4ed937-4ea2-45c9-abaa-31758358b7ea None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:14.530899 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 207/408] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Jul 26 13:33:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:33:14.538929 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4ed937-4ea2-45c9-abaa-31758358b7ea None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:14.592120 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:14.752584 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a687a4d-3caf-4354-bdde-3333d77c5ae8 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:14.779418 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4ed937-4ea2-45c9-abaa-31758358b7ea None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:14.799498 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf70218-f10a-4dc5-b620-d794a5dbdf3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:14.800383 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf70218-f10a-4dc5-b620-d794a5dbdf3a None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:14.800666 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf70218-f10a-4dc5-b620-d794a5dbdf3a None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:14.800936 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf70218-f10a-4dc5-b620-d794a5dbdf3a None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:14.802227 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf70218-f10a-4dc5-b620-d794a5dbdf3a None admin] RBAC: Adding query filter params (name=admin) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:14.802539 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf70218-f10a-4dc5-b620-d794a5dbdf3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:14.804194 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf70218-f10a-4dc5-b620-d794a5dbdf3a None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:14.819839 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 202/409] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Jul 26 13:33:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:33:14.819839 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9995f104-9842-448b-9a20-d08d926512bb None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:14.819839 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9995f104-9842-448b-9a20-d08d926512bb None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:14.819839 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9995f104-9842-448b-9a20-d08d926512bb None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:14.819839 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 203/410] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:33:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:33:14.820757 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9995f104-9842-448b-9a20-d08d926512bb None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:14.848855 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cfcc63-03a4-48fc-8e3f-3144b03a41b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:14.850285 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cfcc63-03a4-48fc-8e3f-3144b03a41b0 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:14.850832 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cfcc63-03a4-48fc-8e3f-3144b03a41b0 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:14.852008 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cfcc63-03a4-48fc-8e3f-3144b03a41b0 None admin] PATH_INFO: `/v3/groups/04331d538f1c4d479d659675e703da04` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:14.854903 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cfcc63-03a4-48fc-8e3f-3144b03a41b0 None admin] RBAC: Authorizing `identity:get_group(group_id=04331d538f1c4d479d659675e703da04)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:14.859216 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cfcc63-03a4-48fc-8e3f-3144b03a41b0 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:14.863590 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 208/411] 10.222.0.58 () {56 vars in 1180 bytes} [Tue Jul 26 13:33:14 2022] GET /identity/v3/groups/04331d538f1c4d479d659675e703da04 => generated 226 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:14.865998 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de8fe92-92f0-456f-b02f-8855c9bb5ab2 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:14.866696 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de8fe92-92f0-456f-b02f-8855c9bb5ab2 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:14.867398 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de8fe92-92f0-456f-b02f-8855c9bb5ab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:14.878658 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:14.895222 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-2de8fe92-92f0-456f-b02f-8855c9bb5ab2 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:33:14.896419 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-2de8fe92-92f0-456f-b02f-8855c9bb5ab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:33:14.967778 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 209/412] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:33:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 13:33:14.968662 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de8fe92-92f0-456f-b02f-8855c9bb5ab2 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:14.980304 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9995f104-9842-448b-9a20-d08d926512bb None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:15.006896 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:15.047263 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c64a91-ccb0-4ea4-b63e-9f94f2f3f1f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:15.048246 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c64a91-ccb0-4ea4-b63e-9f94f2f3f1f2 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:15.048738 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c64a91-ccb0-4ea4-b63e-9f94f2f3f1f2 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:15.049459 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c64a91-ccb0-4ea4-b63e-9f94f2f3f1f2 None admin] PATH_INFO: `/v3/groups/90ce4124fd5f4779a13930568b5e0aae` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:15.050981 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c64a91-ccb0-4ea4-b63e-9f94f2f3f1f2 None admin] RBAC: Authorizing `identity:get_group(group_id=90ce4124fd5f4779a13930568b5e0aae)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:15.052684 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c64a91-ccb0-4ea4-b63e-9f94f2f3f1f2 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:15.054862 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 204/413] 10.222.0.58 () {56 vars in 1180 bytes} [Tue Jul 26 13:33:14 2022] GET /identity/v3/groups/90ce4124fd5f4779a13930568b5e0aae => generated 229 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:33:15.060664 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c64a91-ccb0-4ea4-b63e-9f94f2f3f1f2 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:15.101891 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de8fe92-92f0-456f-b02f-8855c9bb5ab2 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:15.139162 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76050a9e-3d71-4dc2-afc2-62343e42ca3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:15.139370 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76050a9e-3d71-4dc2-afc2-62343e42ca3f None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:15.139613 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76050a9e-3d71-4dc2-afc2-62343e42ca3f None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:15.139883 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76050a9e-3d71-4dc2-afc2-62343e42ca3f None admin] PATH_INFO: `/v3/projects/6ad88492b683452a81ab7150c22b665e` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:15.141061 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76050a9e-3d71-4dc2-afc2-62343e42ca3f None admin] RBAC: Authorizing `identity:get_project(project_id=6ad88492b683452a81ab7150c22b665e)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:15.142521 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76050a9e-3d71-4dc2-afc2-62343e42ca3f None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:15.149247 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 210/414] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Jul 26 13:33:14 2022] GET /identity/v3/projects/6ad88492b683452a81ab7150c22b665e => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:15.150046 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76050a9e-3d71-4dc2-afc2-62343e42ca3f None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:15.172920 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c64a91-ccb0-4ea4-b63e-9f94f2f3f1f2 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:15.181289 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:15.212713 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d790a1-222c-4ab7-be48-830d9dcfe763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:15.213764 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d790a1-222c-4ab7-be48-830d9dcfe763 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:15.214298 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d790a1-222c-4ab7-be48-830d9dcfe763 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:15.214874 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d790a1-222c-4ab7-be48-830d9dcfe763 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:15.226502 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d790a1-222c-4ab7-be48-830d9dcfe763 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:33:15.232264 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-f0d790a1-222c-4ab7-be48-830d9dcfe763 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 13:33:15.232264 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:15.232264 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:15.232264 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:15.232264 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:15.232264 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:15.232264 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:15.232264 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:15.232264 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:15.232264 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:15.232264 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:15.232264 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:15.232264 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:33:15.232264 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:33:15.232264 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:33:15.232264 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:33:15.233134 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:33:15.233134 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:33:15.233134 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:33:15.233134 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:33:15.233134 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:15.233134 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:15.233134 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:15.233134 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:15.233134 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:15.233134 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:15.233134 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:15.233134 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:15.233134 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:15.233134 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:15.233134 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:15.233134 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:15.234022 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:15.234022 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:15.234022 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:15.234022 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:15.234022 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:33:15.234022 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:33:15.234022 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:33:15.234022 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:33:15.234022 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:33:15.234022 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:33:15.234022 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 13:33:15.234022 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:33:15.236667 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 205/415] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Jul 26 13:33:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:15.237658 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d790a1-222c-4ab7-be48-830d9dcfe763 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:15.274803 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:15.293361 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76050a9e-3d71-4dc2-afc2-62343e42ca3f None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:15.333042 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d35d3c3-ea01-421e-842a-1fc2974c596a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:15.333716 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d35d3c3-ea01-421e-842a-1fc2974c596a None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:15.333997 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d35d3c3-ea01-421e-842a-1fc2974c596a None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:15.334266 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d35d3c3-ea01-421e-842a-1fc2974c596a None admin] PATH_INFO: `/v3/projects/2e341f5b1be34081bfdda39547d48a7c` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:15.335445 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d35d3c3-ea01-421e-842a-1fc2974c596a None admin] RBAC: Authorizing `identity:get_project(project_id=2e341f5b1be34081bfdda39547d48a7c)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:15.336892 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d35d3c3-ea01-421e-842a-1fc2974c596a None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:15.338955 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 211/416] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Jul 26 13:33:15 2022] GET /identity/v3/projects/2e341f5b1be34081bfdda39547d48a7c => generated 344 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:15.343755 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d35d3c3-ea01-421e-842a-1fc2974c596a None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:15.357035 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d790a1-222c-4ab7-be48-830d9dcfe763 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:15.397361 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72a95cd-9134-4579-a9ba-dfc5ece479d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:15.398732 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72a95cd-9134-4579-a9ba-dfc5ece479d9 None admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:15.399487 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72a95cd-9134-4579-a9ba-dfc5ece479d9 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:15.400339 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72a95cd-9134-4579-a9ba-dfc5ece479d9 None admin] PATH_INFO: `/v3/projects/6ad88492b683452a81ab7150c22b665e/groups/04331d538f1c4d479d659675e703da04/roles/5cb730ec786743c3bb8af83da90f9309` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:15.402965 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72a95cd-9134-4579-a9ba-dfc5ece479d9 None admin] RBAC: Authorizing `identity:create_grant(project_id=6ad88492b683452a81ab7150c22b665e, group_id=04331d538f1c4d479d659675e703da04, role_id=5cb730ec786743c3bb8af83da90f9309)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:15.404701 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72a95cd-9134-4579-a9ba-dfc5ece479d9 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:15.424641 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 206/417] 10.222.0.58 () {58 vars in 1442 bytes} [Tue Jul 26 13:33:15 2022] PUT /identity/v3/projects/6ad88492b683452a81ab7150c22b665e/groups/04331d538f1c4d479d659675e703da04/roles/5cb730ec786743c3bb8af83da90f9309 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:33:15.425493 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72a95cd-9134-4579-a9ba-dfc5ece479d9 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:15.469696 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:15.470122 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d35d3c3-ea01-421e-842a-1fc2974c596a None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:15.540303 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a104b8f1-05ed-4f82-883d-105d22e5dd90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:15.541399 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a104b8f1-05ed-4f82-883d-105d22e5dd90 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:15.542463 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a104b8f1-05ed-4f82-883d-105d22e5dd90 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:15.542963 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a104b8f1-05ed-4f82-883d-105d22e5dd90 None admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:15.545135 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a104b8f1-05ed-4f82-883d-105d22e5dd90 None admin] RBAC: Adding query filter params (name=member) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:15.545662 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a104b8f1-05ed-4f82-883d-105d22e5dd90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:15.548649 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a104b8f1-05ed-4f82-883d-105d22e5dd90 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:15.565667 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 212/418] 10.222.0.58 () {56 vars in 1101 bytes} [Tue Jul 26 13:33:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:15.567110 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e00380-eaf4-44c4-a3a6-d6554f7e2052 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:15.567398 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e00380-eaf4-44c4-a3a6-d6554f7e2052 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:15.568035 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e00380-eaf4-44c4-a3a6-d6554f7e2052 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:15.569507 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 213/419] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:33:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:33:15.570276 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e00380-eaf4-44c4-a3a6-d6554f7e2052 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:15.594052 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72a95cd-9134-4579-a9ba-dfc5ece479d9 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:15.600020 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:15.641300 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1aff38-2615-417e-a137-3241c91a13b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:15.641930 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1aff38-2615-417e-a137-3241c91a13b7 None admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:15.642675 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1aff38-2615-417e-a137-3241c91a13b7 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:15.642955 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1aff38-2615-417e-a137-3241c91a13b7 None admin] PATH_INFO: `/v3/projects/2e341f5b1be34081bfdda39547d48a7c/groups/90ce4124fd5f4779a13930568b5e0aae/roles/6e0201a9a6484f7d8b112b1eafff70bd` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:15.645221 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1aff38-2615-417e-a137-3241c91a13b7 None admin] RBAC: Authorizing `identity:create_grant(project_id=2e341f5b1be34081bfdda39547d48a7c, group_id=90ce4124fd5f4779a13930568b5e0aae, role_id=6e0201a9a6484f7d8b112b1eafff70bd)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:15.646765 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1aff38-2615-417e-a137-3241c91a13b7 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:15.664930 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 207/420] 10.222.0.58 () {58 vars in 1442 bytes} [Tue Jul 26 13:33:15 2022] PUT /identity/v3/projects/2e341f5b1be34081bfdda39547d48a7c/groups/90ce4124fd5f4779a13930568b5e0aae/roles/6e0201a9a6484f7d8b112b1eafff70bd => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:33:15.666071 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0993947a-3398-4a99-8278-9db1df3a6d86 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:15.666368 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0993947a-3398-4a99-8278-9db1df3a6d86 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:15.666640 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0993947a-3398-4a99-8278-9db1df3a6d86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:15.690983 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-0993947a-3398-4a99-8278-9db1df3a6d86 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:33:15.692237 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-0993947a-3398-4a99-8278-9db1df3a6d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:33:15.703698 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e00380-eaf4-44c4-a3a6-d6554f7e2052 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:15.744596 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aabe634-adb8-423d-acd9-7e8d8bbaedff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:15.745777 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aabe634-adb8-423d-acd9-7e8d8bbaedff None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:15.746301 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aabe634-adb8-423d-acd9-7e8d8bbaedff None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:15.746823 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aabe634-adb8-423d-acd9-7e8d8bbaedff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:15.823418 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 208/421] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:33:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 13:33:15.850028 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aabe634-adb8-423d-acd9-7e8d8bbaedff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:15.850581 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0993947a-3398-4a99-8278-9db1df3a6d86 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:15.852374 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aabe634-adb8-423d-acd9-7e8d8bbaedff None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:15.881906 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:15.906575 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:15.971812 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 214/422] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Jul 26 13:33:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 13:33:15.978366 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aabe634-adb8-423d-acd9-7e8d8bbaedff None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:16.014857 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0993947a-3398-4a99-8278-9db1df3a6d86 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:16.114298 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31728e3b-9359-41e0-9421-b8175934aab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:16.114974 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31728e3b-9359-41e0-9421-b8175934aab9 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:16.115264 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31728e3b-9359-41e0-9421-b8175934aab9 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:16.115588 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31728e3b-9359-41e0-9421-b8175934aab9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:16.122609 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aabe634-adb8-423d-acd9-7e8d8bbaedff None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:16.141763 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31728e3b-9359-41e0-9421-b8175934aab9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:33:16.147189 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:16.150934 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-31728e3b-9359-41e0-9421-b8175934aab9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 13:33:16.150934 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:16.150934 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:16.150934 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:16.150934 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:16.150934 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:16.150934 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:16.150934 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:16.150934 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:16.150934 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:16.150934 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:16.150934 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:16.150934 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:33:16.150934 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:33:16.150934 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:33:16.150934 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:33:16.152494 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:33:16.152494 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:33:16.152494 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:33:16.152494 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:33:16.152494 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:16.152494 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:16.152494 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:16.152494 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:16.152494 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:16.152494 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:16.152494 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:16.152494 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:16.152494 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:16.152494 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:16.152494 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:16.152494 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:16.153267 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:16.153267 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:16.153267 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:16.153267 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:16.153267 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:33:16.153267 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:33:16.153267 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:33:16.153267 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:33:16.153267 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:33:16.153267 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:33:16.153267 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 13:33:16.153267 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:33:16.153267 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 209/423] 10.222.0.58 () {56 vars in 1114 bytes} [Tue Jul 26 13:33:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:16.158750 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31728e3b-9359-41e0-9421-b8175934aab9 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:16.176007 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c56e010-8a8c-4f95-a25a-380d46c5895d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:16.177059 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c56e010-8a8c-4f95-a25a-380d46c5895d None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:16.177594 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c56e010-8a8c-4f95-a25a-380d46c5895d None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:16.179474 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c56e010-8a8c-4f95-a25a-380d46c5895d None admin] PATH_INFO: `/v3/projects/afb9ca9466054e008e3b052866da363d` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:16.181109 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c56e010-8a8c-4f95-a25a-380d46c5895d None admin] RBAC: Authorizing `identity:get_project(project_id=afb9ca9466054e008e3b052866da363d)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:16.183039 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c56e010-8a8c-4f95-a25a-380d46c5895d None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:16.187069 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 215/424] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Jul 26 13:33:15 2022] GET /identity/v3/projects/afb9ca9466054e008e3b052866da363d => generated 298 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:33:16.193830 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c56e010-8a8c-4f95-a25a-380d46c5895d None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:16.327322 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:16.345519 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c56e010-8a8c-4f95-a25a-380d46c5895d None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:16.370387 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31728e3b-9359-41e0-9421-b8175934aab9 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:16.390411 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af4ed8c-a4a5-48ee-8b5c-50f5099bbe2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:16.391951 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af4ed8c-a4a5-48ee-8b5c-50f5099bbe2b None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:16.392494 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af4ed8c-a4a5-48ee-8b5c-50f5099bbe2b None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:16.392984 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af4ed8c-a4a5-48ee-8b5c-50f5099bbe2b None admin] PATH_INFO: `/v3/groups/04331d538f1c4d479d659675e703da04` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:16.394459 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af4ed8c-a4a5-48ee-8b5c-50f5099bbe2b None admin] RBAC: Authorizing `identity:get_group(group_id=04331d538f1c4d479d659675e703da04)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:16.396125 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af4ed8c-a4a5-48ee-8b5c-50f5099bbe2b None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:16.398562 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 216/425] 10.222.0.58 () {56 vars in 1180 bytes} [Tue Jul 26 13:33:16 2022] GET /identity/v3/groups/04331d538f1c4d479d659675e703da04 => generated 226 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:16.422488 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af4ed8c-a4a5-48ee-8b5c-50f5099bbe2b None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:16.424428 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ebdd81-f0c4-4ff8-9478-9602fbc682b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:16.425159 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ebdd81-f0c4-4ff8-9478-9602fbc682b4 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:16.425572 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ebdd81-f0c4-4ff8-9478-9602fbc682b4 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:16.425953 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ebdd81-f0c4-4ff8-9478-9602fbc682b4 None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:16.426778 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ebdd81-f0c4-4ff8-9478-9602fbc682b4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:16.427183 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ebdd81-f0c4-4ff8-9478-9602fbc682b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:16.433501 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ebdd81-f0c4-4ff8-9478-9602fbc682b4 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:16.438721 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 210/426] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Jul 26 13:33:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:33:16.443703 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5f40c6-4296-4180-ab5a-c9638a4773a0 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:16.443994 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5f40c6-4296-4180-ab5a-c9638a4773a0 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:16.444271 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5f40c6-4296-4180-ab5a-c9638a4773a0 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:16.445888 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 211/427] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:33:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:33:16.448670 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5f40c6-4296-4180-ab5a-c9638a4773a0 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:16.497667 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:16.551485 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af4ed8c-a4a5-48ee-8b5c-50f5099bbe2b None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:16.567411 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5f40c6-4296-4180-ab5a-c9638a4773a0 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:16.593557 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a93d122-b54c-469d-ae7f-a60bbc87804e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:16.594755 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a93d122-b54c-469d-ae7f-a60bbc87804e None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:16.595261 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a93d122-b54c-469d-ae7f-a60bbc87804e None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:16.595830 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a93d122-b54c-469d-ae7f-a60bbc87804e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:16.596827 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a93d122-b54c-469d-ae7f-a60bbc87804e None admin] RBAC: Adding query filter params (group.id=04331d538f1c4d479d659675e703da04, role.id=759f389b221049c4b01faf3b984cd8d7, scope.project.id=afb9ca9466054e008e3b052866da363d) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:16.597365 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a93d122-b54c-469d-ae7f-a60bbc87804e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:16.599256 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a93d122-b54c-469d-ae7f-a60bbc87804e None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:16.609534 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf461de0-c356-4664-b1ca-22c67a622cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:16.609534 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf461de0-c356-4664-b1ca-22c67a622cf8 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:16.609534 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf461de0-c356-4664-b1ca-22c67a622cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:16.609534 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf461de0-c356-4664-b1ca-22c67a622cf8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:16.613082 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 217/428] 10.222.0.58 () {56 vars in 1376 bytes} [Tue Jul 26 13:33:16 2022] GET /identity/v3/role_assignments?group.id=04331d538f1c4d479d659675e703da04&scope.project.id=afb9ca9466054e008e3b052866da363d&role.id=759f389b221049c4b01faf3b984cd8d7 => generated 611 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:16.613082 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285a3550-7894-4e06-9dc8-fedd1610deb2 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:16.613276 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285a3550-7894-4e06-9dc8-fedd1610deb2 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:16.613451 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285a3550-7894-4e06-9dc8-fedd1610deb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:16.626098 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:16.658612 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-285a3550-7894-4e06-9dc8-fedd1610deb2 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:33:16.659862 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-285a3550-7894-4e06-9dc8-fedd1610deb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:33:16.767536 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf461de0-c356-4664-b1ca-22c67a622cf8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:16.768940 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf461de0-c356-4664-b1ca-22c67a622cf8 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:16.811854 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 218/429] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:33:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 13:33:16.836701 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285a3550-7894-4e06-9dc8-fedd1610deb2 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:16.896666 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:16.915684 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 212/430] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Jul 26 13:33:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 26 13:33:16.921747 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf461de0-c356-4664-b1ca-22c67a622cf8 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:16.965529 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285a3550-7894-4e06-9dc8-fedd1610deb2 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:16.969978 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:17.006221 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f01032-2448-4666-8e8b-4328a509d1c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:17.007077 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f01032-2448-4666-8e8b-4328a509d1c5 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:17.007451 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f01032-2448-4666-8e8b-4328a509d1c5 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:17.007828 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f01032-2448-4666-8e8b-4328a509d1c5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:17.020539 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f01032-2448-4666-8e8b-4328a509d1c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:33:17.026095 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-68f01032-2448-4666-8e8b-4328a509d1c5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 13:33:17.026095 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:17.026095 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:17.026095 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:17.026095 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:17.026095 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:17.026095 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:17.026095 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:17.026095 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:17.026095 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:17.026095 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:17.026095 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:17.026095 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:33:17.026095 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:33:17.026095 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:33:17.026095 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:33:17.027607 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:33:17.027607 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:33:17.027607 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:33:17.027607 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:33:17.027607 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:17.027607 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:17.027607 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:17.027607 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:17.027607 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:17.027607 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:17.027607 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:17.027607 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:17.027607 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:17.027607 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:17.027607 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:17.027607 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:17.028457 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:17.028457 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:17.028457 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:17.028457 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:17.028457 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:33:17.028457 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:33:17.028457 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:33:17.028457 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:33:17.028457 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:33:17.028457 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:33:17.028457 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 13:33:17.028457 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:33:17.034394 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 219/431] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Jul 26 13:33:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 13:33:17.035019 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f01032-2448-4666-8e8b-4328a509d1c5 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:17.054102 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf461de0-c356-4664-b1ca-22c67a622cf8 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:17.094642 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c4f0b4-4a95-4e6b-961e-2e2f411f1b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:17.095624 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c4f0b4-4a95-4e6b-961e-2e2f411f1b33 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:17.096078 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c4f0b4-4a95-4e6b-961e-2e2f411f1b33 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:17.096640 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c4f0b4-4a95-4e6b-961e-2e2f411f1b33 None admin] PATH_INFO: `/v3/projects/afb9ca9466054e008e3b052866da363d` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:17.098139 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c4f0b4-4a95-4e6b-961e-2e2f411f1b33 None admin] RBAC: Authorizing `identity:get_project(project_id=afb9ca9466054e008e3b052866da363d)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:17.099818 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c4f0b4-4a95-4e6b-961e-2e2f411f1b33 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:17.102278 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 213/432] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Jul 26 13:33:16 2022] GET /identity/v3/projects/afb9ca9466054e008e3b052866da363d => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:17.110141 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c4f0b4-4a95-4e6b-961e-2e2f411f1b33 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:17.181072 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f01032-2448-4666-8e8b-4328a509d1c5 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:17.199201 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:17.235792 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a26694b-eb07-43b0-866d-13dece2d38b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:17.236849 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a26694b-eb07-43b0-866d-13dece2d38b7 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:17.237370 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a26694b-eb07-43b0-866d-13dece2d38b7 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:17.237892 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a26694b-eb07-43b0-866d-13dece2d38b7 None admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:17.238992 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a26694b-eb07-43b0-866d-13dece2d38b7 None admin] RBAC: Adding query filter params (name=member) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:17.239595 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a26694b-eb07-43b0-866d-13dece2d38b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:17.241244 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a26694b-eb07-43b0-866d-13dece2d38b7 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:17.251060 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c4f0b4-4a95-4e6b-961e-2e2f411f1b33 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:17.258269 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 220/433] 10.222.0.58 () {56 vars in 1101 bytes} [Tue Jul 26 13:33:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:17.259541 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:17.261124 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a26694b-eb07-43b0-866d-13dece2d38b7 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:17.297390 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b365f67a-305b-43df-8001-378db3704ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:17.297982 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b365f67a-305b-43df-8001-378db3704ed3 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:17.298268 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b365f67a-305b-43df-8001-378db3704ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:17.298538 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b365f67a-305b-43df-8001-378db3704ed3 None admin] PATH_INFO: `/v3/groups/04331d538f1c4d479d659675e703da04` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:17.300507 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b365f67a-305b-43df-8001-378db3704ed3 None admin] RBAC: Authorizing `identity:get_group(group_id=04331d538f1c4d479d659675e703da04)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:17.302268 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b365f67a-305b-43df-8001-378db3704ed3 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:17.304425 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 214/434] 10.222.0.58 () {56 vars in 1180 bytes} [Tue Jul 26 13:33:17 2022] GET /identity/v3/groups/04331d538f1c4d479d659675e703da04 => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:33:17.310797 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b365f67a-305b-43df-8001-378db3704ed3 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:17.395703 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a26694b-eb07-43b0-866d-13dece2d38b7 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:17.420118 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b365f67a-305b-43df-8001-378db3704ed3 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:17.441916 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b06105d-a5d5-4b8a-bdf0-709e8611b235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:17.443328 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b06105d-a5d5-4b8a-bdf0-709e8611b235 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:17.444111 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b06105d-a5d5-4b8a-bdf0-709e8611b235 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:17.445097 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b06105d-a5d5-4b8a-bdf0-709e8611b235 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:17.457685 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846f9e3d-f873-41ae-9556-c8a77cb96ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:17.458220 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846f9e3d-f873-41ae-9556-c8a77cb96ef8 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:17.458502 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846f9e3d-f873-41ae-9556-c8a77cb96ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:17.458778 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846f9e3d-f873-41ae-9556-c8a77cb96ef8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:17.459581 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846f9e3d-f873-41ae-9556-c8a77cb96ef8 None admin] RBAC: Adding query filter params (group.id=04331d538f1c4d479d659675e703da04, role.id=5cb730ec786743c3bb8af83da90f9309, scope.project.id=afb9ca9466054e008e3b052866da363d) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:17.459893 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846f9e3d-f873-41ae-9556-c8a77cb96ef8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:17.461379 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846f9e3d-f873-41ae-9556-c8a77cb96ef8 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:17.469254 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 215/435] 10.222.0.58 () {56 vars in 1376 bytes} [Tue Jul 26 13:33:17 2022] GET /identity/v3/role_assignments?group.id=04331d538f1c4d479d659675e703da04&scope.project.id=afb9ca9466054e008e3b052866da363d&role.id=5cb730ec786743c3bb8af83da90f9309 => generated 611 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:33:17.497757 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:17.539869 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b06105d-a5d5-4b8a-bdf0-709e8611b235 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:17.542385 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b06105d-a5d5-4b8a-bdf0-709e8611b235 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:17.679251 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 221/436] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Jul 26 13:33:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 13:33:17.685861 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846f9e3d-f873-41ae-9556-c8a77cb96ef8 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:17.745490 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:17.752381 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c888a3-380c-4f1b-9f6a-6851a7b0c794 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:17.752694 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c888a3-380c-4f1b-9f6a-6851a7b0c794 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:17.752943 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c888a3-380c-4f1b-9f6a-6851a7b0c794 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:17.754694 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 222/437] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:33:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:33:17.760262 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e0fd3-0cd5-4f02-96fd-9d1371b972db None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:17.760582 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e0fd3-0cd5-4f02-96fd-9d1371b972db None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:17.760864 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e0fd3-0cd5-4f02-96fd-9d1371b972db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:17.782963 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-da5e0fd3-0cd5-4f02-96fd-9d1371b972db None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:33:17.785843 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-da5e0fd3-0cd5-4f02-96fd-9d1371b972db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:33:17.834838 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846f9e3d-f873-41ae-9556-c8a77cb96ef8 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:17.872927 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:17.880432 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7be2cb-c1fe-45b5-9c52-462adce067e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:17.881143 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7be2cb-c1fe-45b5-9c52-462adce067e2 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:17.882433 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7be2cb-c1fe-45b5-9c52-462adce067e2 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:17.882547 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 223/438] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:33:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 13:33:17.882634 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7be2cb-c1fe-45b5-9c52-462adce067e2 None admin] PATH_INFO: `/v3/projects/6ad88492b683452a81ab7150c22b665e` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:17.883057 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7be2cb-c1fe-45b5-9c52-462adce067e2 None admin] RBAC: Authorizing `identity:get_project(project_id=6ad88492b683452a81ab7150c22b665e)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:17.884600 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7be2cb-c1fe-45b5-9c52-462adce067e2 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:17.891972 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 216/439] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Jul 26 13:33:17 2022] GET /identity/v3/projects/6ad88492b683452a81ab7150c22b665e => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:17.894020 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5e0fd3-0cd5-4f02-96fd-9d1371b972db None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:17.907709 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7be2cb-c1fe-45b5-9c52-462adce067e2 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:18.023483 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5e0fd3-0cd5-4f02-96fd-9d1371b972db None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:18.046215 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7be2cb-c1fe-45b5-9c52-462adce067e2 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:18.056820 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:18.074027 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0c2727-d665-40fe-8a39-b7f04dc6818f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:18.074665 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0c2727-d665-40fe-8a39-b7f04dc6818f None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:18.074946 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0c2727-d665-40fe-8a39-b7f04dc6818f None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:18.075222 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0c2727-d665-40fe-8a39-b7f04dc6818f None admin] PATH_INFO: `/v3/groups/04331d538f1c4d479d659675e703da04` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:18.076418 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0c2727-d665-40fe-8a39-b7f04dc6818f None admin] RBAC: Authorizing `identity:get_group(group_id=04331d538f1c4d479d659675e703da04)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:18.077910 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0c2727-d665-40fe-8a39-b7f04dc6818f None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:18.079543 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 224/440] 10.222.0.58 () {56 vars in 1180 bytes} [Tue Jul 26 13:33:17 2022] GET /identity/v3/groups/04331d538f1c4d479d659675e703da04 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:18.085990 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0c2727-d665-40fe-8a39-b7f04dc6818f None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:18.094540 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b87f80-3b9c-4c6c-ad53-4d4e58f64a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:18.095190 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b87f80-3b9c-4c6c-ad53-4d4e58f64a64 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:18.095472 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b87f80-3b9c-4c6c-ad53-4d4e58f64a64 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:18.095745 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b87f80-3b9c-4c6c-ad53-4d4e58f64a64 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:18.108508 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b87f80-3b9c-4c6c-ad53-4d4e58f64a64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:33:18.113931 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-03b87f80-3b9c-4c6c-ad53-4d4e58f64a64 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 13:33:18.113931 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:18.113931 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:18.113931 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:18.113931 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:18.113931 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:18.113931 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:18.113931 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:18.113931 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:18.113931 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:18.113931 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:18.113931 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:18.113931 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:33:18.113931 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:33:18.113931 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:33:18.113931 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:33:18.115113 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:33:18.115113 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:33:18.115113 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:33:18.115113 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:33:18.115113 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:18.115113 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:18.115113 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:18.115113 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:18.115113 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:18.115113 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:18.115113 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:18.115113 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:18.115113 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:18.115113 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:18.115113 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:18.115113 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:18.115919 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:18.115919 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:18.115919 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:18.115919 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:18.115919 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:33:18.115919 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:33:18.115919 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:33:18.115919 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:33:18.115919 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:33:18.115919 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:33:18.115919 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 13:33:18.115919 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:33:18.115919 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 217/441] 10.222.0.58 () {56 vars in 1114 bytes} [Tue Jul 26 13:33:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:18.121681 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b87f80-3b9c-4c6c-ad53-4d4e58f64a64 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:18.165507 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:18.205728 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0c2727-d665-40fe-8a39-b7f04dc6818f None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:18.237431 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b87f80-3b9c-4c6c-ad53-4d4e58f64a64 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:18.242287 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57e87fa-6452-4f79-b8a1-da636cbca04a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:18.242904 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57e87fa-6452-4f79-b8a1-da636cbca04a None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:18.243184 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57e87fa-6452-4f79-b8a1-da636cbca04a None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:18.243460 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57e87fa-6452-4f79-b8a1-da636cbca04a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:18.244247 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57e87fa-6452-4f79-b8a1-da636cbca04a None admin] RBAC: Adding query filter params (group.id=04331d538f1c4d479d659675e703da04, role.id=759f389b221049c4b01faf3b984cd8d7, scope.project.id=6ad88492b683452a81ab7150c22b665e) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:18.244554 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57e87fa-6452-4f79-b8a1-da636cbca04a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:18.246080 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57e87fa-6452-4f79-b8a1-da636cbca04a None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:18.252011 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 225/442] 10.222.0.58 () {56 vars in 1376 bytes} [Tue Jul 26 13:33:18 2022] GET /identity/v3/role_assignments?group.id=04331d538f1c4d479d659675e703da04&scope.project.id=6ad88492b683452a81ab7150c22b665e&role.id=759f389b221049c4b01faf3b984cd8d7 => generated 611 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:33:18.253213 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98ea39e-62b1-4288-aec8-494a9c2aa3cd None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:18.253522 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98ea39e-62b1-4288-aec8-494a9c2aa3cd None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:18.253797 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98ea39e-62b1-4288-aec8-494a9c2aa3cd None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:18.255032 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 226/443] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:33:18 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:33:18.260759 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59405c74-82ae-4fd6-b235-64da837bdd83 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:18.261061 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59405c74-82ae-4fd6-b235-64da837bdd83 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:18.261322 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59405c74-82ae-4fd6-b235-64da837bdd83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:18.274802 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bcd49b-1a1a-4aad-90b2-f9584a7ff028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:18.275353 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bcd49b-1a1a-4aad-90b2-f9584a7ff028 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:18.275637 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bcd49b-1a1a-4aad-90b2-f9584a7ff028 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:18.276458 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bcd49b-1a1a-4aad-90b2-f9584a7ff028 None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:18.277180 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bcd49b-1a1a-4aad-90b2-f9584a7ff028 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:18.277507 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bcd49b-1a1a-4aad-90b2-f9584a7ff028 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:18.278943 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bcd49b-1a1a-4aad-90b2-f9584a7ff028 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:18.282061 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-59405c74-82ae-4fd6-b235-64da837bdd83 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:33:18.283307 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-59405c74-82ae-4fd6-b235-64da837bdd83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:33:18.293896 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 218/444] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Jul 26 13:33:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:33:18.297408 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bcd49b-1a1a-4aad-90b2-f9584a7ff028 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:18.330461 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:18.371914 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 227/445] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:33:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 13:33:18.404979 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59405c74-82ae-4fd6-b235-64da837bdd83 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:18.441868 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bcd49b-1a1a-4aad-90b2-f9584a7ff028 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:18.482837 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:18.486843 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdf7ab2-e892-468e-aee1-f942ec1cdf52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:18.488535 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf7ab2-e892-468e-aee1-f942ec1cdf52 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:18.488535 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf7ab2-e892-468e-aee1-f942ec1cdf52 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:18.488535 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf7ab2-e892-468e-aee1-f942ec1cdf52 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:18.541985 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59405c74-82ae-4fd6-b235-64da837bdd83 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:18.572030 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdf7ab2-e892-468e-aee1-f942ec1cdf52 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:18.574346 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdf7ab2-e892-468e-aee1-f942ec1cdf52 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:18.586090 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3833be4b-487b-4e99-a6c5-dd21f3a471e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:18.586979 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3833be4b-487b-4e99-a6c5-dd21f3a471e7 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:18.587398 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3833be4b-487b-4e99-a6c5-dd21f3a471e7 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:18.587815 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3833be4b-487b-4e99-a6c5-dd21f3a471e7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:18.600806 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3833be4b-487b-4e99-a6c5-dd21f3a471e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:33:18.605996 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-3833be4b-487b-4e99-a6c5-dd21f3a471e7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 13:33:18.605996 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:18.605996 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:18.605996 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:18.605996 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:18.605996 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:18.605996 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:18.605996 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:18.605996 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:18.605996 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:18.605996 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:18.605996 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:18.605996 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:33:18.605996 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:33:18.605996 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:33:18.605996 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:33:18.607055 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:33:18.607055 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:33:18.607055 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:33:18.607055 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:33:18.607055 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:18.607055 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:18.607055 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:18.607055 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:18.607055 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:18.607055 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:18.607055 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:18.607055 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:18.607055 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:18.607055 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:18.607055 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:18.607055 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:18.607855 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:18.607855 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:18.607855 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:18.607855 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:18.607855 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:33:18.607855 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:33:18.607855 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:33:18.607855 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:33:18.607855 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:33:18.607855 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:33:18.607855 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 13:33:18.607855 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:33:18.607855 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 228/446] 10.222.0.58 () {56 vars in 1096 bytes} [Tue Jul 26 13:33:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 13:33:18.611482 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:18.613282 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3833be4b-487b-4e99-a6c5-dd21f3a471e7 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:18.692131 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 219/447] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Jul 26 13:33:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 13:33:18.697676 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdf7ab2-e892-468e-aee1-f942ec1cdf52 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:18.719587 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3833be4b-487b-4e99-a6c5-dd21f3a471e7 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:18.739658 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:18.756095 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b66e72-832f-4ff9-9fbe-31aa03c57044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:18.756718 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b66e72-832f-4ff9-9fbe-31aa03c57044 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:18.757000 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b66e72-832f-4ff9-9fbe-31aa03c57044 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:18.757270 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b66e72-832f-4ff9-9fbe-31aa03c57044 None admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:18.758021 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b66e72-832f-4ff9-9fbe-31aa03c57044 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:18.758328 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b66e72-832f-4ff9-9fbe-31aa03c57044 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:18.759704 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b66e72-832f-4ff9-9fbe-31aa03c57044 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:18.768692 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 229/448] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Jul 26 13:33:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:18.774308 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b66e72-832f-4ff9-9fbe-31aa03c57044 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:18.809820 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdf7ab2-e892-468e-aee1-f942ec1cdf52 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:18.837061 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:18.845577 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0213f012-6c27-41e3-942f-14f3a1c3c8aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:18.846233 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0213f012-6c27-41e3-942f-14f3a1c3c8aa None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:18.846516 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0213f012-6c27-41e3-942f-14f3a1c3c8aa None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:18.846792 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0213f012-6c27-41e3-942f-14f3a1c3c8aa None admin] PATH_INFO: `/v3/projects/6ad88492b683452a81ab7150c22b665e` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:18.847988 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0213f012-6c27-41e3-942f-14f3a1c3c8aa None admin] RBAC: Authorizing `identity:get_project(project_id=6ad88492b683452a81ab7150c22b665e)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:18.849446 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0213f012-6c27-41e3-942f-14f3a1c3c8aa None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:18.851151 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 220/449] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Jul 26 13:33:18 2022] GET /identity/v3/projects/6ad88492b683452a81ab7150c22b665e => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:33:18.856711 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0213f012-6c27-41e3-942f-14f3a1c3c8aa None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:18.880257 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b66e72-832f-4ff9-9fbe-31aa03c57044 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:18.915693 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02d0616-9fe9-478f-8e30-9c16ea9656a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:18.916460 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02d0616-9fe9-478f-8e30-9c16ea9656a8 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:18.916806 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02d0616-9fe9-478f-8e30-9c16ea9656a8 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:18.917087 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02d0616-9fe9-478f-8e30-9c16ea9656a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:18.960589 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0213f012-6c27-41e3-942f-14f3a1c3c8aa None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:18.967774 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:18.986689 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02d0616-9fe9-478f-8e30-9c16ea9656a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:18.988137 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02d0616-9fe9-478f-8e30-9c16ea9656a8 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:18.994491 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eba920-ba54-4beb-b3ed-3b2b3c2cf6e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:18.995647 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eba920-ba54-4beb-b3ed-3b2b3c2cf6e2 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:18.995925 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eba920-ba54-4beb-b3ed-3b2b3c2cf6e2 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:18.996196 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eba920-ba54-4beb-b3ed-3b2b3c2cf6e2 None admin] PATH_INFO: `/v3/groups/04331d538f1c4d479d659675e703da04` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:18.997366 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1eba920-ba54-4beb-b3ed-3b2b3c2cf6e2 None admin] RBAC: Authorizing `identity:get_group(group_id=04331d538f1c4d479d659675e703da04)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:18.998825 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1eba920-ba54-4beb-b3ed-3b2b3c2cf6e2 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:19.000408 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 221/450] 10.222.0.58 () {56 vars in 1180 bytes} [Tue Jul 26 13:33:18 2022] GET /identity/v3/groups/04331d538f1c4d479d659675e703da04 => generated 226 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:33:19.006163 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eba920-ba54-4beb-b3ed-3b2b3c2cf6e2 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:19.076892 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:19.101829 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 230/451] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Jul 26 13:33:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 26 13:33:19.108010 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02d0616-9fe9-478f-8e30-9c16ea9656a8 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:19.122412 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eba920-ba54-4beb-b3ed-3b2b3c2cf6e2 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:19.157570 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2b8516-dd18-406e-b669-54ab6ce0ff97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:19.158204 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2b8516-dd18-406e-b669-54ab6ce0ff97 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:19.158461 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2b8516-dd18-406e-b669-54ab6ce0ff97 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:19.158739 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2b8516-dd18-406e-b669-54ab6ce0ff97 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:19.159547 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2b8516-dd18-406e-b669-54ab6ce0ff97 None admin] RBAC: Adding query filter params (group.id=04331d538f1c4d479d659675e703da04, role.id=5cb730ec786743c3bb8af83da90f9309, scope.project.id=6ad88492b683452a81ab7150c22b665e) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:19.159855 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2b8516-dd18-406e-b669-54ab6ce0ff97 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:19.161313 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2b8516-dd18-406e-b669-54ab6ce0ff97 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:19.168078 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 222/452] 10.222.0.58 () {56 vars in 1376 bytes} [Tue Jul 26 13:33:19 2022] GET /identity/v3/role_assignments?group.id=04331d538f1c4d479d659675e703da04&scope.project.id=6ad88492b683452a81ab7150c22b665e&role.id=5cb730ec786743c3bb8af83da90f9309 => generated 611 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:33:19.198819 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:19.219894 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02d0616-9fe9-478f-8e30-9c16ea9656a8 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:19.259031 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaa4f6c-b742-4a4b-a00a-7f486d9ac8d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:19.259704 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaa4f6c-b742-4a4b-a00a-7f486d9ac8d2 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:19.259989 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaa4f6c-b742-4a4b-a00a-7f486d9ac8d2 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:19.260300 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaa4f6c-b742-4a4b-a00a-7f486d9ac8d2 None admin] PATH_INFO: `/v3/projects/2e341f5b1be34081bfdda39547d48a7c` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:19.261574 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcaa4f6c-b742-4a4b-a00a-7f486d9ac8d2 None admin] RBAC: Authorizing `identity:get_project(project_id=2e341f5b1be34081bfdda39547d48a7c)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:19.263028 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcaa4f6c-b742-4a4b-a00a-7f486d9ac8d2 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:19.264748 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 231/453] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Jul 26 13:33:19 2022] GET /identity/v3/projects/2e341f5b1be34081bfdda39547d48a7c => generated 344 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:33:19.270558 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2b8516-dd18-406e-b669-54ab6ce0ff97 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:19.386642 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2b8516-dd18-406e-b669-54ab6ce0ff97 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:19.413573 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:19.423774 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d5b3f8-b557-4dc3-8f70-feeb8523f81b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:19.424679 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d5b3f8-b557-4dc3-8f70-feeb8523f81b None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:19.424960 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d5b3f8-b557-4dc3-8f70-feeb8523f81b None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:19.425232 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d5b3f8-b557-4dc3-8f70-feeb8523f81b None admin] PATH_INFO: `/v3/groups/90ce4124fd5f4779a13930568b5e0aae` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:19.426452 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d5b3f8-b557-4dc3-8f70-feeb8523f81b None admin] RBAC: Authorizing `identity:get_group(group_id=90ce4124fd5f4779a13930568b5e0aae)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:19.427994 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d5b3f8-b557-4dc3-8f70-feeb8523f81b None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:19.429637 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 223/454] 10.222.0.58 () {56 vars in 1180 bytes} [Tue Jul 26 13:33:19 2022] GET /identity/v3/groups/90ce4124fd5f4779a13930568b5e0aae => generated 229 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:19.436192 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaa4f6c-b742-4a4b-a00a-7f486d9ac8d2 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:19.542500 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaa4f6c-b742-4a4b-a00a-7f486d9ac8d2 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:19.575644 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fe5cea-16ad-4b26-92f1-66d83f598c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:19.576256 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fe5cea-16ad-4b26-92f1-66d83f598c5d None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:19.576533 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fe5cea-16ad-4b26-92f1-66d83f598c5d None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:19.576802 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fe5cea-16ad-4b26-92f1-66d83f598c5d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:19.577614 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fe5cea-16ad-4b26-92f1-66d83f598c5d None admin] RBAC: Adding query filter params (group.id=90ce4124fd5f4779a13930568b5e0aae, role.id=6e0201a9a6484f7d8b112b1eafff70bd, scope.project.id=2e341f5b1be34081bfdda39547d48a7c) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:19.577923 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fe5cea-16ad-4b26-92f1-66d83f598c5d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:19.579331 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fe5cea-16ad-4b26-92f1-66d83f598c5d None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:19.585262 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 232/455] 10.222.0.58 () {56 vars in 1376 bytes} [Tue Jul 26 13:33:19 2022] GET /identity/v3/role_assignments?group.id=90ce4124fd5f4779a13930568b5e0aae&scope.project.id=2e341f5b1be34081bfdda39547d48a7c&role.id=6e0201a9a6484f7d8b112b1eafff70bd => generated 611 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:22.259303 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02a9419-fb5c-4f7d-b2c7-6996eaad1996 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:22.260819 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02a9419-fb5c-4f7d-b2c7-6996eaad1996 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:22.261303 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02a9419-fb5c-4f7d-b2c7-6996eaad1996 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:22.263147 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 224/456] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:33:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:33:22.278297 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e11a495-e4a8-42f3-a7d1-99279f853e8c None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:22.279263 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e11a495-e4a8-42f3-a7d1-99279f853e8c None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:22.279788 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e11a495-e4a8-42f3-a7d1-99279f853e8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:22.318631 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-8e11a495-e4a8-42f3-a7d1-99279f853e8c None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:33:22.320174 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-8e11a495-e4a8-42f3-a7d1-99279f853e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:33:22.331495 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:22.386886 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50572e1-cc62-4589-a048-1c3b7989b1c2 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:22.387408 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50572e1-cc62-4589-a048-1c3b7989b1c2 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:22.387827 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50572e1-cc62-4589-a048-1c3b7989b1c2 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:22.390729 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 225/457] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:33:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:33:22.395187 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba0347d-2cbb-4e51-a353-73d860160e43 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:22.395910 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba0347d-2cbb-4e51-a353-73d860160e43 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:22.396404 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba0347d-2cbb-4e51-a353-73d860160e43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:22.421028 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-bba0347d-2cbb-4e51-a353-73d860160e43 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:33:22.422642 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-bba0347d-2cbb-4e51-a353-73d860160e43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:33:22.432643 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 233/458] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:33:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 13:33:22.468324 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e11a495-e4a8-42f3-a7d1-99279f853e8c None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:22.500533 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 226/459] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:33:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 13:33:22.525386 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba0347d-2cbb-4e51-a353-73d860160e43 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:22.603906 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e11a495-e4a8-42f3-a7d1-99279f853e8c None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:22.640747 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0338f9a1-a822-4f45-a577-8a5c0e5114ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:22.641977 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0338f9a1-a822-4f45-a577-8a5c0e5114ad None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:22.642474 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0338f9a1-a822-4f45-a577-8a5c0e5114ad None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:22.642953 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0338f9a1-a822-4f45-a577-8a5c0e5114ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:22.647061 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba0347d-2cbb-4e51-a353-73d860160e43 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:22.654930 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0338f9a1-a822-4f45-a577-8a5c0e5114ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:22.656748 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0338f9a1-a822-4f45-a577-8a5c0e5114ad None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:22.659928 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:22.676080 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:22.678348 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-0338f9a1-a822-4f45-a577-8a5c0e5114ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:22.678348 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:22.678348 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:22.678348 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:22.678348 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:22.678348 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:22.678348 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:22.678348 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:22.678348 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:22.678348 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:22.678348 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:22.678348 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:22.678348 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 13:33:22.678348 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 13:33:22.678348 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 13:33:22.679252 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 13:33:22.679252 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:22.679252 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:22.679252 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:22.679252 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:22.679252 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:22.679252 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:22.679252 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:22.679252 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:22.679252 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:22.679252 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:22.679252 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:22.679252 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:22.679252 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:22.679252 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:22.679252 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:22.679252 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:22.680100 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 13:33:22.680100 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 13:33:22.680100 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:22.680100 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:33:22.684418 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 234/460] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:33:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 13:33:22.685949 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bfbc89-74ac-4a4d-881c-6cd405209540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:22.686535 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f231775a-8abf-4b1a-9342-a5bc20370cce None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:22.686932 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f231775a-8abf-4b1a-9342-a5bc20370cce None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:22.687180 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bfbc89-74ac-4a4d-881c-6cd405209540 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:22.687393 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f231775a-8abf-4b1a-9342-a5bc20370cce None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:22.687751 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bfbc89-74ac-4a4d-881c-6cd405209540 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:22.688165 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bfbc89-74ac-4a4d-881c-6cd405209540 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:22.689641 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 235/461] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:33:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:33:22.690459 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f231775a-8abf-4b1a-9342-a5bc20370cce None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:22.700712 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bfbc89-74ac-4a4d-881c-6cd405209540 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:22.702440 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bfbc89-74ac-4a4d-881c-6cd405209540 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:22.718105 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-06bfbc89-74ac-4a4d-881c-6cd405209540 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:22.718105 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:22.718105 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:22.718105 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:22.718105 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:22.718105 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:22.718105 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:22.718105 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:22.718105 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:22.718105 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:22.718105 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:22.718105 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:22.718105 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 13:33:22.718105 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 13:33:22.718105 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 13:33:22.719009 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 13:33:22.719009 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:22.719009 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:22.719009 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:22.719009 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:22.719009 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:22.719009 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:22.719009 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:22.719009 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:22.719009 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:22.719009 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:22.719009 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:22.719009 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:22.719009 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:22.719009 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:22.719009 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:22.719009 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:22.719860 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 13:33:22.719860 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 13:33:22.719860 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:22.719860 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:33:22.725168 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 227/462] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:33:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:22.725168 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b360774-7355-4f41-86f3-f5a6cd9a0f45 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:22.725168 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b360774-7355-4f41-86f3-f5a6cd9a0f45 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:22.725168 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b360774-7355-4f41-86f3-f5a6cd9a0f45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:22.742540 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-1b360774-7355-4f41-86f3-f5a6cd9a0f45 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:33:22.743734 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-1b360774-7355-4f41-86f3-f5a6cd9a0f45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:33:22.813933 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f231775a-8abf-4b1a-9342-a5bc20370cce None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:22.819635 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 228/463] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:33:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 13:33:22.821122 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b804331-14cf-41d9-a0ab-eeb7ba5634fe None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:22.821411 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b804331-14cf-41d9-a0ab-eeb7ba5634fe None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:22.821727 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b804331-14cf-41d9-a0ab-eeb7ba5634fe None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:22.823166 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 229/464] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:33:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:33:22.823801 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b804331-14cf-41d9-a0ab-eeb7ba5634fe None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:22.861073 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bbdec1-9da2-40bc-af94-4799b3b5a81c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:22.861796 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bbdec1-9da2-40bc-af94-4799b3b5a81c None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:22.862085 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bbdec1-9da2-40bc-af94-4799b3b5a81c None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:22.862364 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bbdec1-9da2-40bc-af94-4799b3b5a81c None admin] PATH_INFO: `/v3/domains` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:22.863071 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bbdec1-9da2-40bc-af94-4799b3b5a81c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:22.863384 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bbdec1-9da2-40bc-af94-4799b3b5a81c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:22.864780 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bbdec1-9da2-40bc-af94-4799b3b5a81c None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:22.891559 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 236/465] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Jul 26 13:33:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:22.893219 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06008e02-764e-4ea8-86ac-c3f9cd032396 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:22.893563 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06008e02-764e-4ea8-86ac-c3f9cd032396 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:22.893897 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06008e02-764e-4ea8-86ac-c3f9cd032396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:22.918656 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-06008e02-764e-4ea8-86ac-c3f9cd032396 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:33:22.919893 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-06008e02-764e-4ea8-86ac-c3f9cd032396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:33:22.947071 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b804331-14cf-41d9-a0ab-eeb7ba5634fe None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:22.956940 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:22.981545 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:22.982956 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4515ce-34d4-41a2-8852-3e211654159b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:22.983576 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4515ce-34d4-41a2-8852-3e211654159b None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:22.983891 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4515ce-34d4-41a2-8852-3e211654159b None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:22.984244 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4515ce-34d4-41a2-8852-3e211654159b None admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:22.985591 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4515ce-34d4-41a2-8852-3e211654159b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:22.985591 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4515ce-34d4-41a2-8852-3e211654159b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:22.986986 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4515ce-34d4-41a2-8852-3e211654159b None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:22.999513 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 230/466] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Jul 26 13:33:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:23.000351 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4515ce-34d4-41a2-8852-3e211654159b None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:23.006015 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 237/467] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:33:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 13:33:23.007362 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06008e02-764e-4ea8-86ac-c3f9cd032396 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:23.124705 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06008e02-764e-4ea8-86ac-c3f9cd032396 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:23.134022 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4515ce-34d4-41a2-8852-3e211654159b None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:23.164031 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345c9c87-3f48-495e-ae5f-7f6d0d779861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:23.165004 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345c9c87-3f48-495e-ae5f-7f6d0d779861 None admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:23.165418 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345c9c87-3f48-495e-ae5f-7f6d0d779861 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:23.165906 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345c9c87-3f48-495e-ae5f-7f6d0d779861 None admin] PATH_INFO: `/v3/users` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:23.167290 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345c9c87-3f48-495e-ae5f-7f6d0d779861 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:23.169352 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345c9c87-3f48-495e-ae5f-7f6d0d779861 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:23.174478 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a229a10-aef2-45e8-a954-f3a24a12ebeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:23.175125 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a229a10-aef2-45e8-a954-f3a24a12ebeb None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:23.175408 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a229a10-aef2-45e8-a954-f3a24a12ebeb None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:23.175680 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a229a10-aef2-45e8-a954-f3a24a12ebeb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:23.191759 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a229a10-aef2-45e8-a954-f3a24a12ebeb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:23.193274 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a229a10-aef2-45e8-a954-f3a24a12ebeb None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:23.195609 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 238/468] 10.222.0.58 () {60 vars in 1132 bytes} [Tue Jul 26 13:33:23 2022] POST /identity/v3/users => generated 246 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:33:23.197189 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345c9c87-3f48-495e-ae5f-7f6d0d779861 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:23.211632 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-8a229a10-aef2-45e8-a954-f3a24a12ebeb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:23.211632 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:23.211632 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:23.211632 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:23.211632 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:23.211632 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:23.211632 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:23.211632 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:23.211632 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:23.211632 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:23.211632 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:23.211632 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:23.211632 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 13:33:23.211632 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 13:33:23.211632 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 13:33:23.212581 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 13:33:23.212581 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:23.212581 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:23.212581 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:23.212581 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:23.212581 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:23.212581 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:23.212581 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:23.212581 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:23.212581 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:23.212581 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:23.212581 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:23.212581 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:23.212581 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:23.212581 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:23.212581 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:23.212581 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:23.213969 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 13:33:23.213969 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 13:33:23.213969 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:23.213969 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:33:23.218692 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 231/469] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:33:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:23.219805 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a229a10-aef2-45e8-a954-f3a24a12ebeb None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:23.254453 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:23.273997 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:23.358055 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a229a10-aef2-45e8-a954-f3a24a12ebeb None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:23.362169 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345c9c87-3f48-495e-ae5f-7f6d0d779861 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:23.399080 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a63619c-9a0f-415f-838e-18c10050690c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:23.399984 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a63619c-9a0f-415f-838e-18c10050690c None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:23.400441 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a63619c-9a0f-415f-838e-18c10050690c None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:23.401557 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a63619c-9a0f-415f-838e-18c10050690c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:23.401991 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083cd220-9b39-468d-8378-b902463f1de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:23.402694 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083cd220-9b39-468d-8378-b902463f1de8 None admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:23.403058 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083cd220-9b39-468d-8378-b902463f1de8 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:23.403405 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083cd220-9b39-468d-8378-b902463f1de8 None admin] PATH_INFO: `/v3/users` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:23.405050 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083cd220-9b39-468d-8378-b902463f1de8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:23.406545 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083cd220-9b39-468d-8378-b902463f1de8 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:23.415038 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a63619c-9a0f-415f-838e-18c10050690c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:23.417664 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a63619c-9a0f-415f-838e-18c10050690c None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:23.431076 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 239/470] 10.222.0.58 () {60 vars in 1132 bytes} [Tue Jul 26 13:33:23 2022] POST /identity/v3/users => generated 248 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:33:23.431945 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083cd220-9b39-468d-8378-b902463f1de8 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:23.444899 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-4a63619c-9a0f-415f-838e-18c10050690c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:23.444899 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:23.444899 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:23.444899 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:23.444899 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:23.444899 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:23.444899 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:23.444899 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:23.444899 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:23.444899 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:23.444899 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:23.444899 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:23.444899 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 13:33:23.444899 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 13:33:23.444899 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 13:33:23.445864 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 13:33:23.445864 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:23.445864 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:23.445864 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:23.445864 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:23.445864 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:23.445864 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:23.445864 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:23.445864 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:23.445864 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:23.445864 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:23.445864 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:23.445864 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:23.445864 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:23.445864 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:23.445864 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:23.445864 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:23.446716 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 13:33:23.446716 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 13:33:23.446716 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:23.446716 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:33:23.449219 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 232/471] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:33:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:23.457090 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a63619c-9a0f-415f-838e-18c10050690c None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:23.565168 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:23.581281 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083cd220-9b39-468d-8378-b902463f1de8 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:23.608987 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a63619c-9a0f-415f-838e-18c10050690c None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:23.613890 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:23.636068 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4755b92e-9cbb-42e0-a550-49e7f7fddedf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:23.636709 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4755b92e-9cbb-42e0-a550-49e7f7fddedf None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:23.636995 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4755b92e-9cbb-42e0-a550-49e7f7fddedf None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:23.637273 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4755b92e-9cbb-42e0-a550-49e7f7fddedf None admin] PATH_INFO: `/v3/domains` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:23.638080 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4755b92e-9cbb-42e0-a550-49e7f7fddedf None admin] RBAC: Adding query filter params (name=Default) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:23.639001 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4755b92e-9cbb-42e0-a550-49e7f7fddedf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:23.641654 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4755b92e-9cbb-42e0-a550-49e7f7fddedf None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:23.656992 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 240/472] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Jul 26 13:33:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:23.659255 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f74183-a963-46b6-a584-fbe33b13271b None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:23.659685 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f74183-a963-46b6-a584-fbe33b13271b None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:23.660043 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f74183-a963-46b6-a584-fbe33b13271b None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:23.661513 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e783432a-5a8f-499e-8c96-0bcde537cdae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:23.661691 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 241/473] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:33:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:33:23.662146 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e783432a-5a8f-499e-8c96-0bcde537cdae None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:23.662422 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e783432a-5a8f-499e-8c96-0bcde537cdae None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:23.662694 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e783432a-5a8f-499e-8c96-0bcde537cdae None admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:23.663390 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e783432a-5a8f-499e-8c96-0bcde537cdae None admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:23.663695 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e783432a-5a8f-499e-8c96-0bcde537cdae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:23.665072 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e783432a-5a8f-499e-8c96-0bcde537cdae None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:23.665262 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f74183-a963-46b6-a584-fbe33b13271b None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:23.687210 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 233/474] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Jul 26 13:33:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:33:23.687210 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604ad4e3-07e3-4781-906b-bc3fa1fa0ce2 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:23.687210 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604ad4e3-07e3-4781-906b-bc3fa1fa0ce2 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:23.687210 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604ad4e3-07e3-4781-906b-bc3fa1fa0ce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:23.705547 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-604ad4e3-07e3-4781-906b-bc3fa1fa0ce2 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:33:23.706863 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-604ad4e3-07e3-4781-906b-bc3fa1fa0ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:33:23.800718 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 234/475] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:33:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 13:33:23.802161 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604ad4e3-07e3-4781-906b-bc3fa1fa0ce2 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:23.844579 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f74183-a963-46b6-a584-fbe33b13271b None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:23.890651 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83c7eee-e28a-4b76-9ae8-70def8e40ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:23.891306 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83c7eee-e28a-4b76-9ae8-70def8e40ea8 None admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:23.891580 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83c7eee-e28a-4b76-9ae8-70def8e40ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:23.891857 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83c7eee-e28a-4b76-9ae8-70def8e40ea8 None admin] PATH_INFO: `/v3/users` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:23.892845 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83c7eee-e28a-4b76-9ae8-70def8e40ea8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:23.894341 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83c7eee-e28a-4b76-9ae8-70def8e40ea8 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:23.895362 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:23.931095 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 242/476] 10.222.0.58 () {60 vars in 1132 bytes} [Tue Jul 26 13:33:23 2022] POST /identity/v3/users => generated 248 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:33:23.932741 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83c7eee-e28a-4b76-9ae8-70def8e40ea8 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:23.967985 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604ad4e3-07e3-4781-906b-bc3fa1fa0ce2 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:23.977904 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:24.058534 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d82dd1-6a66-44f0-8ee7-261c0a040cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:24.059684 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d82dd1-6a66-44f0-8ee7-261c0a040cf1 None admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:24.060171 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d82dd1-6a66-44f0-8ee7-261c0a040cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:24.060660 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d82dd1-6a66-44f0-8ee7-261c0a040cf1 None admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:24.061618 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d82dd1-6a66-44f0-8ee7-261c0a040cf1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:24.063557 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d82dd1-6a66-44f0-8ee7-261c0a040cf1 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:24.098995 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 235/477] 10.222.0.58 () {60 vars in 1132 bytes} [Tue Jul 26 13:33:23 2022] POST /identity/v3/users => generated 249 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:33:24.115299 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83c7eee-e28a-4b76-9ae8-70def8e40ea8 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:24.155488 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8030ecf-b63d-4282-9046-265e03569f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:24.156148 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8030ecf-b63d-4282-9046-265e03569f40 None admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:24.156434 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8030ecf-b63d-4282-9046-265e03569f40 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:24.156714 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8030ecf-b63d-4282-9046-265e03569f40 None admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:24.157686 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8030ecf-b63d-4282-9046-265e03569f40 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:24.159105 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8030ecf-b63d-4282-9046-265e03569f40 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:24.169492 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.sql.core [None req-c8030ecf-b63d-4282-9046-265e03569f40 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 26 13:33:24.169492 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 26 13:33:24.169492 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [parameters: {'id': 'ac567232140a4a11874c6b475d577e8a', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 26 13:33:24.169492 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=86933) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 26 13:33:24.189463 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-c8030ecf-b63d-4282-9046-265e03569f40 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 26 13:33:24.189463 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:24.189463 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 26 13:33:24.189463 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 26 13:33:24.189463 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 26 13:33:24.189463 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 26 13:33:24.189463 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 26 13:33:24.189463 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application result = self._query(query) Jul 26 13:33:24.189463 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 26 13:33:24.189463 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application conn.query(q) Jul 26 13:33:24.189463 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 26 13:33:24.189463 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 26 13:33:24.189463 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 26 13:33:24.189463 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application result.read() Jul 26 13:33:24.189463 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 26 13:33:24.190387 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 26 13:33:24.190387 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 26 13:33:24.190387 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 26 13:33:24.190387 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 26 13:33:24.190387 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 26 13:33:24.190387 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 26 13:33:24.190387 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 26 13:33:24.190387 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 26 13:33:24.190387 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:33:24.190387 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 26 13:33:24.190387 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:33:24.190387 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:24.190387 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 26 13:33:24.190387 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 26 13:33:24.190387 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 26 13:33:24.190387 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return ref.to_dict() Jul 26 13:33:24.190387 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 26 13:33:24.190387 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application next(self.gen) Jul 26 13:33:24.191286 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 26 13:33:24.191286 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application yield resource Jul 26 13:33:24.191286 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 26 13:33:24.191286 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application next(self.gen) Jul 26 13:33:24.191286 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 26 13:33:24.191286 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application self.session.rollback() Jul 26 13:33:24.191286 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 26 13:33:24.191286 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application self.force_reraise() Jul 26 13:33:24.191286 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 26 13:33:24.191286 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise self.value Jul 26 13:33:24.191286 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 26 13:33:24.191286 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 26 13:33:24.191286 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 26 13:33:24.191286 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application session.commit() Jul 26 13:33:24.191286 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 26 13:33:24.191286 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 26 13:33:24.191286 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 26 13:33:24.192213 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application self._prepare_impl() Jul 26 13:33:24.192213 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 26 13:33:24.192213 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application self.session.flush() Jul 26 13:33:24.192213 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 26 13:33:24.192213 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application self._flush(objects) Jul 26 13:33:24.192213 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 26 13:33:24.192213 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 26 13:33:24.192213 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 26 13:33:24.192213 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application compat.raise_( Jul 26 13:33:24.192213 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 26 13:33:24.192213 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception Jul 26 13:33:24.192213 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 26 13:33:24.192213 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application flush_context.execute() Jul 26 13:33:24.192213 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 26 13:33:24.192213 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rec.execute(self) Jul 26 13:33:24.192213 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 26 13:33:24.192213 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 26 13:33:24.193084 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 26 13:33:24.193084 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 26 13:33:24.193084 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 26 13:33:24.193084 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 26 13:33:24.193084 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 26 13:33:24.193084 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 26 13:33:24.193084 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 26 13:33:24.193084 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 26 13:33:24.193084 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 26 13:33:24.193084 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 26 13:33:24.193084 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 26 13:33:24.193084 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 26 13:33:24.193084 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 26 13:33:24.193084 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 26 13:33:24.193084 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 26 13:33:24.193084 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception Jul 26 13:33:24.195415 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 26 13:33:24.195415 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 26 13:33:24.195415 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 26 13:33:24.195415 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 26 13:33:24.195415 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 26 13:33:24.195415 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application result = self._query(query) Jul 26 13:33:24.195415 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 26 13:33:24.195415 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application conn.query(q) Jul 26 13:33:24.195415 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 26 13:33:24.195415 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 26 13:33:24.195415 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 26 13:33:24.195415 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application result.read() Jul 26 13:33:24.195415 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 26 13:33:24.195415 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 26 13:33:24.195415 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 26 13:33:24.195415 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 26 13:33:24.198252 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 26 13:33:24.198252 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 26 13:33:24.198252 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 26 13:33:24.198252 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 26 13:33:24.198252 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 26 13:33:24.198252 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 26 13:33:24.198252 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [parameters: {'id': 'ac567232140a4a11874c6b475d577e8a', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 26 13:33:24.198252 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 26 13:33:24.198252 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:33:24.198252 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 13:33:24.198252 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:33:24.198252 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:24.198252 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:24.198252 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:24.198252 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:24.198252 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:24.200161 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:24.200161 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:24.200161 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:24.200161 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:24.200161 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:24.200161 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:24.200161 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 26 13:33:24.200161 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 26 13:33:24.200161 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:24.200161 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:24.200161 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 26 13:33:24.200161 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 26 13:33:24.200161 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 26 13:33:24.200161 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 26 13:33:24.200161 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 26 13:33:24.200161 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:33:24.202109 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 243/478] 10.222.0.58 () {60 vars in 1131 bytes} [Tue Jul 26 13:33:23 2022] POST /identity/v3/roles => generated 144 bytes in 259 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:24.209851 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d82dd1-6a66-44f0-8ee7-261c0a040cf1 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:24.348675 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d82dd1-6a66-44f0-8ee7-261c0a040cf1 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:24.370612 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:24.393422 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d95cef-1589-4ebd-9aae-51003304ab3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:24.394114 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d95cef-1589-4ebd-9aae-51003304ab3c None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:24.394394 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d95cef-1589-4ebd-9aae-51003304ab3c None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:24.394669 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d95cef-1589-4ebd-9aae-51003304ab3c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:24.407385 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d95cef-1589-4ebd-9aae-51003304ab3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:33:24.413749 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-d6d95cef-1589-4ebd-9aae-51003304ab3c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 13:33:24.413749 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:24.413749 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:24.413749 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:24.413749 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:24.413749 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:24.413749 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:24.413749 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:24.413749 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:24.413749 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:24.413749 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:24.413749 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:24.413749 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:33:24.413749 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:33:24.413749 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:33:24.413749 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:33:24.415799 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:33:24.415799 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:33:24.415799 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:33:24.415799 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:33:24.415799 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:24.415799 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:24.415799 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:24.415799 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:24.415799 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:24.415799 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:24.415799 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:24.415799 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:24.415799 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:24.415799 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:24.415799 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:24.415799 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:24.418334 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:24.418334 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:24.418334 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:24.418334 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:24.418334 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:33:24.418334 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:33:24.418334 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:33:24.418334 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:33:24.418334 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:33:24.418334 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:33:24.418334 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 13:33:24.418334 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:33:24.418334 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 236/479] 10.222.0.58 () {56 vars in 1114 bytes} [Tue Jul 26 13:33:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:24.426620 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8030ecf-b63d-4282-9046-265e03569f40 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:24.545255 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:24.574951 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8030ecf-b63d-4282-9046-265e03569f40 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:24.628670 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07efecac-3508-40ac-b4fc-dd3192917d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:24.629637 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07efecac-3508-40ac-b4fc-dd3192917d3f None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:24.630122 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07efecac-3508-40ac-b4fc-dd3192917d3f None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:24.630567 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07efecac-3508-40ac-b4fc-dd3192917d3f None admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:24.631463 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07efecac-3508-40ac-b4fc-dd3192917d3f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:24.631983 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07efecac-3508-40ac-b4fc-dd3192917d3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:24.633604 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07efecac-3508-40ac-b4fc-dd3192917d3f None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:24.645445 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 244/480] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Jul 26 13:33:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:25.681335 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9423d558-7d18-40a9-9419-b47393f30ea6 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:25.682392 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9423d558-7d18-40a9-9419-b47393f30ea6 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:25.682465 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9423d558-7d18-40a9-9419-b47393f30ea6 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:25.683994 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 237/481] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:33:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:33:25.689383 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d32502-86c2-45d3-aa12-6b0baf10bfe7 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:25.689857 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d32502-86c2-45d3-aa12-6b0baf10bfe7 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:25.690037 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d32502-86c2-45d3-aa12-6b0baf10bfe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:25.720208 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-f3d32502-86c2-45d3-aa12-6b0baf10bfe7 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:33:25.721439 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-f3d32502-86c2-45d3-aa12-6b0baf10bfe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:33:25.816751 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 245/482] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:33:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 13:33:25.851785 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9423d558-7d18-40a9-9419-b47393f30ea6 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:25.950378 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f599d195-bd21-41e6-9f44-95c72adc887f None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:25.951097 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f599d195-bd21-41e6-9f44-95c72adc887f None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:25.951569 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f599d195-bd21-41e6-9f44-95c72adc887f None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:25.953505 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 246/483] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:33:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 13:33:25.958960 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93c2c33-9f48-44c9-82ae-fb68e2bd813c None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:25.959522 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93c2c33-9f48-44c9-82ae-fb68e2bd813c None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:25.959990 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93c2c33-9f48-44c9-82ae-fb68e2bd813c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:25.982975 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-b93c2c33-9f48-44c9-82ae-fb68e2bd813c None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:33:25.985436 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-b93c2c33-9f48-44c9-82ae-fb68e2bd813c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:33:26.016693 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9423d558-7d18-40a9-9419-b47393f30ea6 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:26.079384 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a0b222-c657-4389-97e6-b0aaaff05faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:26.080525 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a0b222-c657-4389-97e6-b0aaaff05faf None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:26.080947 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a0b222-c657-4389-97e6-b0aaaff05faf None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:26.081227 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a0b222-c657-4389-97e6-b0aaaff05faf None admin] PATH_INFO: `/v3/roles/service` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:26.118155 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a0b222-c657-4389-97e6-b0aaaff05faf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:33:26.135510 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-69a0b222-c657-4389-97e6-b0aaaff05faf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 13:33:26.135510 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:26.135510 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:26.135510 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:26.135510 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:26.135510 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:26.135510 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:26.135510 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:26.135510 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:26.135510 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:26.135510 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:26.135510 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:26.135510 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:33:26.135510 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:33:26.135510 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:33:26.135510 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:33:26.136473 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:33:26.136473 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:33:26.136473 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:33:26.136473 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:33:26.136473 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:26.136473 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:26.136473 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:26.136473 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:26.136473 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:26.136473 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:26.136473 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:26.136473 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:26.136473 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:26.136473 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:26.136473 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:26.136473 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:26.137256 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:26.137256 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:26.137256 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:26.137256 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:26.137256 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:33:26.137256 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:33:26.137256 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:33:26.137256 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:33:26.137256 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:33:26.137256 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:33:26.137256 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 13:33:26.137256 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:33:26.137256 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 238/484] 10.222.0.58 () {56 vars in 1102 bytes} [Tue Jul 26 13:33:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:26.139786 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:26.146110 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a0b222-c657-4389-97e6-b0aaaff05faf None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:26.147261 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 247/485] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:33:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 13:33:26.208450 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:26.224105 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93c2c33-9f48-44c9-82ae-fb68e2bd813c None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:26.343841 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a0b222-c657-4389-97e6-b0aaaff05faf None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:26.351913 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93c2c33-9f48-44c9-82ae-fb68e2bd813c None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:26.393102 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0c589e-5b4a-46ca-9173-f5bf9b515854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:26.393845 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c589e-5b4a-46ca-9173-f5bf9b515854 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:26.394126 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c589e-5b4a-46ca-9173-f5bf9b515854 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:26.394402 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c589e-5b4a-46ca-9173-f5bf9b515854 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:26.408210 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0c589e-5b4a-46ca-9173-f5bf9b515854 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:33:26.414066 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-4d0c589e-5b4a-46ca-9173-f5bf9b515854 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 13:33:26.414066 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:26.414066 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:26.414066 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:26.414066 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:26.414066 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:26.414066 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:26.414066 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:26.414066 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:26.414066 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:26.414066 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:26.414066 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:26.414066 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:33:26.414066 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:33:26.414066 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:33:26.414066 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:33:26.416139 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:33:26.416139 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:33:26.416139 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:33:26.416139 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:33:26.416139 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:26.416139 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:26.416139 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:26.416139 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:26.416139 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:26.416139 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:26.416139 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:26.416139 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:26.416139 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:26.416139 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:26.416139 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:26.416139 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:26.418145 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:26.418145 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:26.418145 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:26.418145 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:26.418145 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:33:26.418145 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:33:26.418145 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:33:26.418145 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:33:26.418145 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:33:26.418145 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:33:26.418145 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 13:33:26.418145 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:33:26.418145 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 248/486] 10.222.0.58 () {56 vars in 1102 bytes} [Tue Jul 26 13:33:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 13:33:26.425514 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0c589e-5b4a-46ca-9173-f5bf9b515854 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:26.466895 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76ff009-b385-4f82-b747-8d0ea3cfe3db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:26.467451 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76ff009-b385-4f82-b747-8d0ea3cfe3db None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:26.467732 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76ff009-b385-4f82-b747-8d0ea3cfe3db None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:26.467967 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76ff009-b385-4f82-b747-8d0ea3cfe3db None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:26.468714 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76ff009-b385-4f82-b747-8d0ea3cfe3db None admin] RBAC: Adding query filter params (name=service) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:26.469024 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76ff009-b385-4f82-b747-8d0ea3cfe3db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:26.470449 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76ff009-b385-4f82-b747-8d0ea3cfe3db None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:26.488659 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 239/487] 10.222.0.58 () {56 vars in 1103 bytes} [Tue Jul 26 13:33:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:26.491172 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802614c5-a683-432a-a335-911f1576ce56 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:26.491690 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802614c5-a683-432a-a335-911f1576ce56 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:26.492167 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802614c5-a683-432a-a335-911f1576ce56 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:26.495488 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:26.498960 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 240/488] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:33:26 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:33:26.500492 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802614c5-a683-432a-a335-911f1576ce56 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:26.556133 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0c589e-5b4a-46ca-9173-f5bf9b515854 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:26.595232 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0268e034-8084-4976-9abf-960cfed708d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:26.595863 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0268e034-8084-4976-9abf-960cfed708d8 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:26.596147 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0268e034-8084-4976-9abf-960cfed708d8 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:26.596425 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0268e034-8084-4976-9abf-960cfed708d8 None admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:26.597158 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0268e034-8084-4976-9abf-960cfed708d8 None admin] RBAC: Adding query filter params (name=service) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:26.597467 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0268e034-8084-4976-9abf-960cfed708d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:26.598867 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0268e034-8084-4976-9abf-960cfed708d8 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:26.612973 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 249/489] 10.222.0.58 () {56 vars in 1103 bytes} [Tue Jul 26 13:33:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:26.612973 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8ab515-fc8d-4964-899b-3087d4e7ee38 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:26.612973 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8ab515-fc8d-4964-899b-3087d4e7ee38 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:26.612973 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8ab515-fc8d-4964-899b-3087d4e7ee38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:26.636440 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-6d8ab515-fc8d-4964-899b-3087d4e7ee38 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:33:26.637740 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-6d8ab515-fc8d-4964-899b-3087d4e7ee38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:33:26.655991 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:26.662912 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802614c5-a683-432a-a335-911f1576ce56 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:26.709063 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986f7778-cb0b-4ccc-8d7b-7387195c9f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:26.710560 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986f7778-cb0b-4ccc-8d7b-7387195c9f7f None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:26.711035 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986f7778-cb0b-4ccc-8d7b-7387195c9f7f None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:26.711637 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986f7778-cb0b-4ccc-8d7b-7387195c9f7f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:26.725160 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 250/490] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:33:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 13:33:26.726186 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8ab515-fc8d-4964-899b-3087d4e7ee38 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:26.727569 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986f7778-cb0b-4ccc-8d7b-7387195c9f7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:26.730797 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986f7778-cb0b-4ccc-8d7b-7387195c9f7f None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:26.751919 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-986f7778-cb0b-4ccc-8d7b-7387195c9f7f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:26.751919 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:26.751919 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:26.751919 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:26.751919 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:26.751919 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:26.751919 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:26.751919 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:26.751919 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:26.751919 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:26.751919 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:26.751919 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:26.751919 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 13:33:26.751919 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 13:33:26.751919 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 13:33:26.752821 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 13:33:26.752821 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:26.752821 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:26.752821 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:26.752821 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:26.752821 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:26.752821 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:26.752821 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:26.752821 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:26.752821 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:26.752821 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:26.752821 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:26.752821 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:26.752821 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:26.752821 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:26.752821 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:26.752821 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:26.753686 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 13:33:26.753686 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 13:33:26.753686 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:26.753686 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:33:26.753686 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 241/491] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:33:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:26.754444 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160fb36c-6bbb-4161-a047-8968ed5974ce None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:26.754605 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160fb36c-6bbb-4161-a047-8968ed5974ce None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:26.754881 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160fb36c-6bbb-4161-a047-8968ed5974ce None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:26.760755 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 242/492] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:33:26 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:33:26.761439 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160fb36c-6bbb-4161-a047-8968ed5974ce None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:26.800770 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:26.852416 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8ab515-fc8d-4964-899b-3087d4e7ee38 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:26.880107 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160fb36c-6bbb-4161-a047-8968ed5974ce None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:26.888403 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfe366c-2fbb-475c-84fa-2e9bb535e13b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:26.889176 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfe366c-2fbb-475c-84fa-2e9bb535e13b None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:26.889461 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfe366c-2fbb-475c-84fa-2e9bb535e13b None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:26.889762 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfe366c-2fbb-475c-84fa-2e9bb535e13b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:26.902829 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfe366c-2fbb-475c-84fa-2e9bb535e13b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:26.904303 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfe366c-2fbb-475c-84fa-2e9bb535e13b None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:26.921017 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-cbfe366c-2fbb-475c-84fa-2e9bb535e13b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:26.921017 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:26.921017 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:26.921017 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:26.921017 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:26.921017 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:26.921017 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:26.921017 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:26.921017 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:26.921017 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:26.921017 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:26.921017 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:26.921017 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 13:33:26.921017 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 13:33:26.921017 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 13:33:26.922926 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 13:33:26.922926 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:26.922926 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:26.922926 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:26.922926 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:26.922926 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:26.922926 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:26.922926 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:26.922926 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:26.922926 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:26.922926 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:26.922926 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:26.922926 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:26.922926 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:26.922926 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:26.922926 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:26.922926 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:26.924866 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 13:33:26.924866 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 13:33:26.924866 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:26.924866 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:33:26.924866 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 251/493] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:33:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:26.924866 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfe366c-2fbb-475c-84fa-2e9bb535e13b None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:26.927857 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca2b214-104a-47d4-819e-2e687b371468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:26.928643 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca2b214-104a-47d4-819e-2e687b371468 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:26.929048 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca2b214-104a-47d4-819e-2e687b371468 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:26.929330 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca2b214-104a-47d4-819e-2e687b371468 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:26.947147 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca2b214-104a-47d4-819e-2e687b371468 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:33:26.953126 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-bca2b214-104a-47d4-819e-2e687b371468 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 13:33:26.953126 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:26.953126 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:26.953126 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:26.953126 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:26.953126 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:26.953126 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:26.953126 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:26.953126 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:26.953126 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:26.953126 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:26.953126 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:26.953126 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:33:26.953126 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:33:26.953126 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:33:26.953126 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:33:26.954192 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:33:26.954192 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:33:26.954192 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:33:26.954192 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:33:26.954192 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:26.954192 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:26.954192 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:26.954192 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:26.954192 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:26.954192 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:26.954192 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:26.954192 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:26.954192 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:26.954192 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:26.954192 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:26.954192 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:26.954993 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:26.954993 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:26.954993 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:26.954993 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:26.954993 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:33:26.954993 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:33:26.954993 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:33:26.954993 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:33:26.954993 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:33:26.954993 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:33:26.954993 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 13:33:26.954993 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:33:26.954993 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 243/494] 10.222.0.58 () {56 vars in 1102 bytes} [Tue Jul 26 13:33:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:26.956459 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0c0760-cf4a-409f-abdc-7aecfad0482e None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:26.956459 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0c0760-cf4a-409f-abdc-7aecfad0482e None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:26.956459 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0c0760-cf4a-409f-abdc-7aecfad0482e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:26.975464 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-ef0c0760-cf4a-409f-abdc-7aecfad0482e None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:33:26.976629 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-ef0c0760-cf4a-409f-abdc-7aecfad0482e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:33:26.985541 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:27.051658 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfe366c-2fbb-475c-84fa-2e9bb535e13b None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:27.055216 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:27.069541 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 244/495] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:33:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 13:33:27.069541 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0c0760-cf4a-409f-abdc-7aecfad0482e None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:27.107114 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cdb04b-6e85-4d99-9d45-e08884be9939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:27.107721 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cdb04b-6e85-4d99-9d45-e08884be9939 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:27.108006 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cdb04b-6e85-4d99-9d45-e08884be9939 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:27.108349 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cdb04b-6e85-4d99-9d45-e08884be9939 None admin] PATH_INFO: `/v3/domains` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:27.109063 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cdb04b-6e85-4d99-9d45-e08884be9939 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:27.109373 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cdb04b-6e85-4d99-9d45-e08884be9939 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:27.110804 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cdb04b-6e85-4d99-9d45-e08884be9939 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:27.128892 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 252/496] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Jul 26 13:33:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:27.129817 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cdb04b-6e85-4d99-9d45-e08884be9939 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:27.192067 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0c0760-cf4a-409f-abdc-7aecfad0482e None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:27.233080 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4773c0-a109-4c08-93e0-c7f84979c3e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:27.234215 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4773c0-a109-4c08-93e0-c7f84979c3e7 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:27.234500 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4773c0-a109-4c08-93e0-c7f84979c3e7 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:27.235937 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4773c0-a109-4c08-93e0-c7f84979c3e7 None admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:27.237121 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4773c0-a109-4c08-93e0-c7f84979c3e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:27.237468 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4773c0-a109-4c08-93e0-c7f84979c3e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:27.239071 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4773c0-a109-4c08-93e0-c7f84979c3e7 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:27.251322 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cdb04b-6e85-4d99-9d45-e08884be9939 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:27.251720 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 245/497] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Jul 26 13:33:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:27.257063 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4773c0-a109-4c08-93e0-c7f84979c3e7 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:27.260524 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:27.295174 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aee2556-aa09-444a-a934-49591b7ff485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:27.295174 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aee2556-aa09-444a-a934-49591b7ff485 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:27.295448 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aee2556-aa09-444a-a934-49591b7ff485 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:27.296363 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aee2556-aa09-444a-a934-49591b7ff485 None admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:27.297109 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aee2556-aa09-444a-a934-49591b7ff485 None admin] RBAC: Adding query filter params (name=service) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:27.297417 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aee2556-aa09-444a-a934-49591b7ff485 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:27.299089 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aee2556-aa09-444a-a934-49591b7ff485 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:27.308381 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 253/498] 10.222.0.58 () {56 vars in 1103 bytes} [Tue Jul 26 13:33:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:27.309225 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aee2556-aa09-444a-a934-49591b7ff485 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:27.344170 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:27.381523 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4773c0-a109-4c08-93e0-c7f84979c3e7 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:27.424475 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cd4ba0-6245-44a2-87d9-e226a511a6dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:27.425228 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cd4ba0-6245-44a2-87d9-e226a511a6dd None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:27.425616 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cd4ba0-6245-44a2-87d9-e226a511a6dd None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:27.425957 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cd4ba0-6245-44a2-87d9-e226a511a6dd None admin] PATH_INFO: `/v3/roles/service` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:27.433065 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aee2556-aa09-444a-a934-49591b7ff485 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:27.439422 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cd4ba0-6245-44a2-87d9-e226a511a6dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:33:27.446029 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-f4cd4ba0-6245-44a2-87d9-e226a511a6dd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 13:33:27.446029 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:27.446029 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:27.446029 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:27.446029 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:27.446029 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:27.446029 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:27.446029 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:27.446029 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:27.446029 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:27.446029 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:27.446029 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:27.446029 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:33:27.446029 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:33:27.446029 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:33:27.446029 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:33:27.447112 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:33:27.447112 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:33:27.447112 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:33:27.447112 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:33:27.447112 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:27.447112 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:27.447112 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:27.447112 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:27.447112 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:27.447112 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:27.447112 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:27.447112 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:27.447112 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:27.447112 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:27.447112 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:27.447112 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:27.447943 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:27.447943 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:27.447943 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:27.447943 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:27.447943 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:33:27.447943 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:33:27.447943 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:33:27.447943 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:33:27.447943 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:33:27.447943 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:33:27.447943 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 13:33:27.447943 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:33:27.447943 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 246/499] 10.222.0.58 () {56 vars in 1102 bytes} [Tue Jul 26 13:33:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:27.448595 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cd4ba0-6245-44a2-87d9-e226a511a6dd None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:27.475969 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b06cecc-16e0-4b53-969a-518887f77505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:27.476784 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b06cecc-16e0-4b53-969a-518887f77505 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:27.477070 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b06cecc-16e0-4b53-969a-518887f77505 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:27.477349 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b06cecc-16e0-4b53-969a-518887f77505 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:27.486542 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b06cecc-16e0-4b53-969a-518887f77505 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:27.487978 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b06cecc-16e0-4b53-969a-518887f77505 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:27.501831 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-9b06cecc-16e0-4b53-969a-518887f77505 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 26 13:33:27.501831 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:27.501831 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:27.501831 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:27.501831 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:27.501831 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:27.501831 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:27.501831 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:27.501831 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:27.501831 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:27.501831 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:27.501831 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:27.501831 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 13:33:27.501831 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 13:33:27.501831 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 13:33:27.501831 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 13:33:27.503061 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:27.503061 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:27.503061 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 13:33:27.503061 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 13:33:27.503061 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 13:33:27.503061 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 13:33:27.503061 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:27.503061 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:27.503061 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:27.503061 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:27.503061 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:27.503061 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:27.503061 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:27.503061 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:27.503061 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:27.503061 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:27.503061 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:27.503061 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:27.504305 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:27.504305 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:27.504305 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 13:33:27.504305 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 13:33:27.504305 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 13:33:27.504305 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 13:33:27.504305 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 13:33:27.504305 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 13:33:27.504305 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 26 13:33:27.504305 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:33:27.504305 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 254/500] 10.222.0.58 () {56 vars in 1093 bytes} [Tue Jul 26 13:33:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:27.504305 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b06cecc-16e0-4b53-969a-518887f77505 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:27.519220 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:27.562045 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cd4ba0-6245-44a2-87d9-e226a511a6dd None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:27.596503 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:27.597577 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa915337-e528-4ab5-92b2-afa3dfebcc9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:27.598140 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa915337-e528-4ab5-92b2-afa3dfebcc9f None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:27.598451 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa915337-e528-4ab5-92b2-afa3dfebcc9f None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:27.598795 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa915337-e528-4ab5-92b2-afa3dfebcc9f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:27.607379 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa915337-e528-4ab5-92b2-afa3dfebcc9f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:27.608831 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa915337-e528-4ab5-92b2-afa3dfebcc9f None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:27.613949 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b06cecc-16e0-4b53-969a-518887f77505 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:27.622535 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-fa915337-e528-4ab5-92b2-afa3dfebcc9f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 26 13:33:27.622535 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:27.622535 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:27.622535 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:27.622535 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:27.622535 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:27.622535 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:27.622535 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:27.622535 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:27.622535 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:27.622535 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:27.622535 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:27.622535 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 13:33:27.622535 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 13:33:27.622535 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 13:33:27.622535 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 13:33:27.623623 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:27.623623 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:27.623623 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 13:33:27.623623 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 13:33:27.623623 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 13:33:27.623623 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 13:33:27.623623 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:27.623623 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:27.623623 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:27.623623 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:27.623623 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:27.623623 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:27.623623 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:27.623623 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:27.623623 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:27.623623 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:27.623623 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:27.623623 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:27.624525 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:27.624525 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:27.624525 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 13:33:27.624525 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 13:33:27.624525 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 13:33:27.624525 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 13:33:27.624525 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 13:33:27.624525 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 13:33:27.624525 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 26 13:33:27.624525 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:33:27.624525 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 247/501] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Jul 26 13:33:27 2022] GET /identity/v3/users/cinder => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:27.624525 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa915337-e528-4ab5-92b2-afa3dfebcc9f None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:27.650885 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fa07c8-09b7-4c56-9d57-fbf713ac4e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:27.651553 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fa07c8-09b7-4c56-9d57-fbf713ac4e0a None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:27.651833 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fa07c8-09b7-4c56-9d57-fbf713ac4e0a None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:27.652107 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fa07c8-09b7-4c56-9d57-fbf713ac4e0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:27.662907 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fa07c8-09b7-4c56-9d57-fbf713ac4e0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:27.664384 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fa07c8-09b7-4c56-9d57-fbf713ac4e0a None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:27.680623 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-b3fa07c8-09b7-4c56-9d57-fbf713ac4e0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:27.680623 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:27.680623 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:27.680623 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:27.680623 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:27.680623 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:27.680623 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:27.680623 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:27.680623 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:27.680623 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:27.680623 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:27.680623 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:27.680623 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 13:33:27.680623 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 13:33:27.680623 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 13:33:27.682295 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 13:33:27.682295 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:27.682295 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:27.682295 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:27.682295 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:27.682295 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:27.682295 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:27.682295 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:27.682295 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:27.682295 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:27.682295 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:27.682295 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:27.682295 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:27.682295 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:27.682295 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:27.682295 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:27.682295 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:27.683143 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 13:33:27.683143 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 13:33:27.683143 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:27.683143 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:33:27.683143 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 255/502] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:33:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:27.683920 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2daee2-ac2a-43b2-a91c-e7ab030466a6 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:27.684204 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2daee2-ac2a-43b2-a91c-e7ab030466a6 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:27.684478 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2daee2-ac2a-43b2-a91c-e7ab030466a6 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:27.685910 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 256/503] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:33:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:33:27.688614 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2daee2-ac2a-43b2-a91c-e7ab030466a6 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:27.740456 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa915337-e528-4ab5-92b2-afa3dfebcc9f None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:27.763064 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:27.775331 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eab16de-ca31-41f6-87e0-54f6d4e08026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:27.776463 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eab16de-ca31-41f6-87e0-54f6d4e08026 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:27.776743 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eab16de-ca31-41f6-87e0-54f6d4e08026 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:27.777010 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eab16de-ca31-41f6-87e0-54f6d4e08026 None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:27.777749 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eab16de-ca31-41f6-87e0-54f6d4e08026 None admin] RBAC: Adding query filter params (name=service) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:27.778049 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eab16de-ca31-41f6-87e0-54f6d4e08026 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:27.779428 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eab16de-ca31-41f6-87e0-54f6d4e08026 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:27.787715 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 248/504] 10.222.0.58 () {56 vars in 1103 bytes} [Tue Jul 26 13:33:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:27.788545 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eab16de-ca31-41f6-87e0-54f6d4e08026 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:27.809132 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2daee2-ac2a-43b2-a91c-e7ab030466a6 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:27.844549 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af190fa2-ccf7-493a-bb17-d36c19ab1e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:27.845171 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af190fa2-ccf7-493a-bb17-d36c19ab1e10 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:27.845452 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af190fa2-ccf7-493a-bb17-d36c19ab1e10 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:27.845759 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af190fa2-ccf7-493a-bb17-d36c19ab1e10 None admin] PATH_INFO: `/v3/users` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:27.846552 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af190fa2-ccf7-493a-bb17-d36c19ab1e10 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:27.846860 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af190fa2-ccf7-493a-bb17-d36c19ab1e10 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:27.848294 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af190fa2-ccf7-493a-bb17-d36c19ab1e10 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:27.867947 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:27.888872 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 257/505] 10.222.0.58 () {56 vars in 1133 bytes} [Tue Jul 26 13:33:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:27.890254 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af190fa2-ccf7-493a-bb17-d36c19ab1e10 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:27.897447 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eab16de-ca31-41f6-87e0-54f6d4e08026 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:27.933218 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cab86c-5408-4fa5-a148-581ac319bb8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:27.933871 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cab86c-5408-4fa5-a148-581ac319bb8c None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:27.934154 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cab86c-5408-4fa5-a148-581ac319bb8c None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:27.934533 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cab86c-5408-4fa5-a148-581ac319bb8c None admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:27.935334 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cab86c-5408-4fa5-a148-581ac319bb8c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:27.935646 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cab86c-5408-4fa5-a148-581ac319bb8c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:27.937077 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cab86c-5408-4fa5-a148-581ac319bb8c None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:27.975820 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 249/506] 10.222.0.58 () {56 vars in 1137 bytes} [Tue Jul 26 13:33:27 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:27.977096 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee6a7ee-7c35-46aa-bc5f-1be179ffbfd7 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:27.977382 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee6a7ee-7c35-46aa-bc5f-1be179ffbfd7 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:27.977689 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee6a7ee-7c35-46aa-bc5f-1be179ffbfd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:27.995926 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-bee6a7ee-7c35-46aa-bc5f-1be179ffbfd7 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:33:27.997043 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-bee6a7ee-7c35-46aa-bc5f-1be179ffbfd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:33:28.000871 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af190fa2-ccf7-493a-bb17-d36c19ab1e10 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:28.033877 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:28.035519 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e0a218-5d86-4000-9af0-5e53b086d9c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:28.036227 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e0a218-5d86-4000-9af0-5e53b086d9c1 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:28.037254 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e0a218-5d86-4000-9af0-5e53b086d9c1 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:28.037254 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e0a218-5d86-4000-9af0-5e53b086d9c1 None admin] PATH_INFO: `/v3/domains` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:28.037556 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e0a218-5d86-4000-9af0-5e53b086d9c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:28.037906 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e0a218-5d86-4000-9af0-5e53b086d9c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:28.039193 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e0a218-5d86-4000-9af0-5e53b086d9c1 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:28.053508 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 258/507] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Jul 26 13:33:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:28.054354 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e0a218-5d86-4000-9af0-5e53b086d9c1 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:28.071008 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 250/508] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:33:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 13:33:28.071008 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee6a7ee-7c35-46aa-bc5f-1be179ffbfd7 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:28.128691 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:28.174692 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e0a218-5d86-4000-9af0-5e53b086d9c1 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:28.175209 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee6a7ee-7c35-46aa-bc5f-1be179ffbfd7 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:28.210507 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0276412-47e2-4540-9c35-3b5f9561144d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:28.211153 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0276412-47e2-4540-9c35-3b5f9561144d None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:28.211439 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0276412-47e2-4540-9c35-3b5f9561144d None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:28.211716 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0276412-47e2-4540-9c35-3b5f9561144d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:28.211996 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133968c7-6bdf-4d79-bad7-e447c2b6dc29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:28.212630 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133968c7-6bdf-4d79-bad7-e447c2b6dc29 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:28.213530 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133968c7-6bdf-4d79-bad7-e447c2b6dc29 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:28.213802 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133968c7-6bdf-4d79-bad7-e447c2b6dc29 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:28.223678 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0276412-47e2-4540-9c35-3b5f9561144d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:28.225153 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0276412-47e2-4540-9c35-3b5f9561144d None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:28.241585 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-c0276412-47e2-4540-9c35-3b5f9561144d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:28.241585 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:28.241585 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:28.241585 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:28.241585 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:28.241585 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:28.241585 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:28.241585 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:28.241585 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:28.241585 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:28.241585 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:28.241585 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:28.241585 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 13:33:28.241585 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 13:33:28.241585 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 13:33:28.242533 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 13:33:28.242533 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:28.242533 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:28.242533 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:28.242533 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:28.242533 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:28.242533 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:28.242533 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:28.242533 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:28.242533 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:28.242533 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:28.242533 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:28.242533 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:28.242533 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:28.242533 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:28.242533 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:28.242533 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:28.243375 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 13:33:28.243375 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 13:33:28.243375 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:28.243375 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:33:28.244226 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 259/509] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:33:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:28.245067 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0276412-47e2-4540-9c35-3b5f9561144d None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:28.261116 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:28.287954 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133968c7-6bdf-4d79-bad7-e447c2b6dc29 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:28.289424 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133968c7-6bdf-4d79-bad7-e447c2b6dc29 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:28.350360 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0276412-47e2-4540-9c35-3b5f9561144d None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:28.360184 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:28.386372 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e275bd2-1906-4e27-beb2-543db90d11c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:28.386996 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e275bd2-1906-4e27-beb2-543db90d11c5 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:28.387275 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e275bd2-1906-4e27-beb2-543db90d11c5 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:28.387549 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e275bd2-1906-4e27-beb2-543db90d11c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:28.395863 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 251/510] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Jul 26 13:33:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 13:33:28.396730 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133968c7-6bdf-4d79-bad7-e447c2b6dc29 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:28.464325 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e275bd2-1906-4e27-beb2-543db90d11c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:28.465820 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e275bd2-1906-4e27-beb2-543db90d11c5 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:28.477285 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:28.508794 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133968c7-6bdf-4d79-bad7-e447c2b6dc29 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:28.546123 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f09dd7-30dd-499b-975e-9116283fb91f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:28.546664 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f09dd7-30dd-499b-975e-9116283fb91f None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:28.546948 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f09dd7-30dd-499b-975e-9116283fb91f None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:28.547226 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f09dd7-30dd-499b-975e-9116283fb91f None admin] PATH_INFO: `/v3/users/glance` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:28.555757 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f09dd7-30dd-499b-975e-9116283fb91f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:28.557749 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f09dd7-30dd-499b-975e-9116283fb91f None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:28.569677 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-89f09dd7-30dd-499b-975e-9116283fb91f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 26 13:33:28.569677 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:28.569677 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:28.569677 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:28.569677 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:28.569677 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:28.569677 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:28.569677 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:28.569677 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:28.569677 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:28.569677 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:28.569677 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:28.569677 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 13:33:28.569677 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 13:33:28.569677 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 13:33:28.569677 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 13:33:28.570598 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:28.570598 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:28.570598 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 13:33:28.570598 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 13:33:28.570598 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 13:33:28.570598 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 13:33:28.570598 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:28.570598 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:28.570598 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:28.570598 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:28.570598 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:28.570598 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:28.570598 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:28.570598 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:28.570598 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:28.570598 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:28.570598 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:28.570598 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:28.571480 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:28.571480 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:28.571480 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 13:33:28.571480 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 13:33:28.571480 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 13:33:28.571480 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 13:33:28.571480 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 13:33:28.571480 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 13:33:28.571480 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 26 13:33:28.571480 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:33:28.571480 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 252/511] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Jul 26 13:33:28 2022] GET /identity/v3/users/glance => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:28.572022 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f09dd7-30dd-499b-975e-9116283fb91f None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:28.587368 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 260/512] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Jul 26 13:33:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 13:33:28.588208 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e275bd2-1906-4e27-beb2-543db90d11c5 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:28.600300 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:28.685074 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f09dd7-30dd-499b-975e-9116283fb91f None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:28.698946 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e275bd2-1906-4e27-beb2-543db90d11c5 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:28.715755 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:28.720887 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ad0cf9-b725-4ed3-85ca-7eeeb6da61e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:28.721751 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ad0cf9-b725-4ed3-85ca-7eeeb6da61e4 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:28.722132 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ad0cf9-b725-4ed3-85ca-7eeeb6da61e4 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:28.722534 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ad0cf9-b725-4ed3-85ca-7eeeb6da61e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:28.734392 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560f47d8-9ef3-4446-8a63-269be6e1beb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:28.735021 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560f47d8-9ef3-4446-8a63-269be6e1beb1 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:28.735304 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560f47d8-9ef3-4446-8a63-269be6e1beb1 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:28.735580 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560f47d8-9ef3-4446-8a63-269be6e1beb1 None admin] PATH_INFO: `/v3/domains` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:28.735671 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ad0cf9-b725-4ed3-85ca-7eeeb6da61e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:28.736281 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560f47d8-9ef3-4446-8a63-269be6e1beb1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:28.736590 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560f47d8-9ef3-4446-8a63-269be6e1beb1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:28.737109 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ad0cf9-b725-4ed3-85ca-7eeeb6da61e4 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:28.737988 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560f47d8-9ef3-4446-8a63-269be6e1beb1 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:28.750439 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 261/513] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Jul 26 13:33:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:28.751102 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560f47d8-9ef3-4446-8a63-269be6e1beb1 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:28.752608 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-20ad0cf9-b725-4ed3-85ca-7eeeb6da61e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:28.752608 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:28.752608 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:28.752608 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:28.752608 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:28.752608 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:28.752608 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:28.752608 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:28.752608 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:28.752608 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:28.752608 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:28.752608 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:28.752608 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 13:33:28.752608 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 13:33:28.752608 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 13:33:28.753808 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 13:33:28.753808 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:28.753808 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:28.753808 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:28.753808 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:28.753808 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:28.753808 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:28.753808 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:28.753808 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:28.753808 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:28.753808 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:28.753808 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:28.753808 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:28.753808 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:28.753808 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:28.753808 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:28.753808 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:28.754650 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 13:33:28.754650 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 13:33:28.754650 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:28.754650 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:33:28.754650 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 253/514] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:33:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:28.754650 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ad0cf9-b725-4ed3-85ca-7eeeb6da61e4 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:28.841190 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:28.865604 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ad0cf9-b725-4ed3-85ca-7eeeb6da61e4 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:28.878676 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560f47d8-9ef3-4446-8a63-269be6e1beb1 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:28.903447 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a64b3ad-db3b-43d2-a5c6-f3917b4b1ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:28.904246 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a64b3ad-db3b-43d2-a5c6-f3917b4b1ad3 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:28.904590 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a64b3ad-db3b-43d2-a5c6-f3917b4b1ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:28.904962 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a64b3ad-db3b-43d2-a5c6-f3917b4b1ad3 None admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:28.906132 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a64b3ad-db3b-43d2-a5c6-f3917b4b1ad3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:28.906524 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a64b3ad-db3b-43d2-a5c6-f3917b4b1ad3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:28.908315 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a64b3ad-db3b-43d2-a5c6-f3917b4b1ad3 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:28.913780 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e51b9a-b912-4705-9e2e-d9330a306173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:28.914435 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e51b9a-b912-4705-9e2e-d9330a306173 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:28.914719 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e51b9a-b912-4705-9e2e-d9330a306173 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:28.914996 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e51b9a-b912-4705-9e2e-d9330a306173 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:28.926862 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e51b9a-b912-4705-9e2e-d9330a306173 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:28.927616 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 254/515] 10.222.0.58 () {56 vars in 1137 bytes} [Tue Jul 26 13:33:28 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:28.928318 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e51b9a-b912-4705-9e2e-d9330a306173 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:28.928482 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a64b3ad-db3b-43d2-a5c6-f3917b4b1ad3 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:28.946307 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-99e51b9a-b912-4705-9e2e-d9330a306173 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:28.946307 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:28.946307 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:28.946307 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:28.946307 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:28.946307 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:28.946307 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:28.946307 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:28.946307 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:28.946307 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:28.946307 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:28.946307 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:28.946307 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 13:33:28.946307 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 13:33:28.946307 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 13:33:28.947230 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 13:33:28.947230 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:28.947230 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:28.947230 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:28.947230 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:28.947230 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:28.947230 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:28.947230 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:28.947230 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:28.947230 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:28.947230 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:28.947230 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:28.947230 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:28.947230 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:28.947230 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:28.947230 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:28.947230 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:28.948199 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 13:33:28.948199 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 13:33:28.948199 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:28.948199 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:33:28.948199 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 262/516] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:33:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:28.948469 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e51b9a-b912-4705-9e2e-d9330a306173 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:28.973365 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:29.035216 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a64b3ad-db3b-43d2-a5c6-f3917b4b1ad3 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:29.060143 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e51b9a-b912-4705-9e2e-d9330a306173 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:29.075552 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f9670a-405b-4c51-9c3b-3f64b397d42e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:29.076197 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f9670a-405b-4c51-9c3b-3f64b397d42e None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:29.076481 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f9670a-405b-4c51-9c3b-3f64b397d42e None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:29.076765 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f9670a-405b-4c51-9c3b-3f64b397d42e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:29.080912 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:29.087312 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f9670a-405b-4c51-9c3b-3f64b397d42e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:29.088820 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f9670a-405b-4c51-9c3b-3f64b397d42e None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:29.095575 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3c0107-23a6-4994-965d-00e51dbb60e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:29.096185 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3c0107-23a6-4994-965d-00e51dbb60e4 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:29.096474 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3c0107-23a6-4994-965d-00e51dbb60e4 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:29.096746 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3c0107-23a6-4994-965d-00e51dbb60e4 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:29.105198 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-50f9670a-405b-4c51-9c3b-3f64b397d42e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:29.105198 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:29.105198 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:29.105198 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:29.105198 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:29.105198 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:29.105198 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:29.105198 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:29.105198 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:29.105198 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:29.105198 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:29.105198 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:29.105198 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 13:33:29.105198 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 13:33:29.105198 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 13:33:29.106260 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 13:33:29.106260 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:29.106260 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:29.106260 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:29.106260 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:29.106260 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:29.106260 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:29.106260 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:29.106260 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:29.106260 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:29.106260 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:29.106260 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:29.106260 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:29.106260 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:29.106260 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:29.106260 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:29.106260 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:29.107301 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 13:33:29.107301 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 13:33:29.107301 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:29.107301 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:33:29.107556 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3c0107-23a6-4994-965d-00e51dbb60e4 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:29.107556 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3c0107-23a6-4994-965d-00e51dbb60e4 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:29.109071 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 255/517] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:33:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:29.110001 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f9670a-405b-4c51-9c3b-3f64b397d42e None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:29.122296 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-3c3c0107-23a6-4994-965d-00e51dbb60e4 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 26 13:33:29.122296 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:29.122296 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:29.122296 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:29.122296 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:29.122296 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:29.122296 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:29.122296 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:29.122296 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:29.122296 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:29.122296 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:29.122296 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:29.122296 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 13:33:29.122296 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 13:33:29.122296 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 13:33:29.122296 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 13:33:29.124297 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:29.124297 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:29.124297 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 13:33:29.124297 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 13:33:29.124297 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 13:33:29.124297 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 13:33:29.124297 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:29.124297 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:29.124297 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:29.124297 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:29.124297 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:29.124297 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:29.124297 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:29.124297 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:29.124297 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:29.124297 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:29.124297 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:29.124297 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:29.126550 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:29.126550 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:29.126550 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 13:33:29.126550 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 13:33:29.126550 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 13:33:29.126550 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 13:33:29.126550 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 13:33:29.126550 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 13:33:29.126550 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 26 13:33:29.126550 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:33:29.126550 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 263/518] 10.222.0.58 () {56 vars in 1102 bytes} [Tue Jul 26 13:33:28 2022] GET /identity/v3/users/neutron => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:29.126550 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3c0107-23a6-4994-965d-00e51dbb60e4 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:29.214187 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:29.221097 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f9670a-405b-4c51-9c3b-3f64b397d42e None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:29.235150 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3c0107-23a6-4994-965d-00e51dbb60e4 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:29.257015 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8c048a-4500-43f0-83f4-7676c3d55d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:29.257662 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8c048a-4500-43f0-83f4-7676c3d55d28 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:29.257942 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8c048a-4500-43f0-83f4-7676c3d55d28 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:29.258210 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8c048a-4500-43f0-83f4-7676c3d55d28 None admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:29.258899 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8c048a-4500-43f0-83f4-7676c3d55d28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:29.259203 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8c048a-4500-43f0-83f4-7676c3d55d28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:29.260603 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8c048a-4500-43f0-83f4-7676c3d55d28 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:29.272065 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b88c231-2f32-4de5-9230-f623fa53abdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:29.272720 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 256/519] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Jul 26 13:33:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:29.272829 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b88c231-2f32-4de5-9230-f623fa53abdb None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:29.272955 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b88c231-2f32-4de5-9230-f623fa53abdb None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:29.273232 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b88c231-2f32-4de5-9230-f623fa53abdb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:29.273466 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8c048a-4500-43f0-83f4-7676c3d55d28 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:29.326837 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:29.348284 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b88c231-2f32-4de5-9230-f623fa53abdb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:29.349674 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b88c231-2f32-4de5-9230-f623fa53abdb None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:29.381284 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8c048a-4500-43f0-83f4-7676c3d55d28 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:29.416807 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70851588-43ce-4855-b025-8c5afdee8a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:29.417720 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70851588-43ce-4855-b025-8c5afdee8a9d None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:29.418001 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70851588-43ce-4855-b025-8c5afdee8a9d None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:29.418276 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70851588-43ce-4855-b025-8c5afdee8a9d None admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:29.418973 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70851588-43ce-4855-b025-8c5afdee8a9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:29.419275 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70851588-43ce-4855-b025-8c5afdee8a9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:29.420754 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70851588-43ce-4855-b025-8c5afdee8a9d None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:29.432759 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 257/520] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Jul 26 13:33:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:29.433622 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70851588-43ce-4855-b025-8c5afdee8a9d None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:29.452210 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:29.458435 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 264/521] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Jul 26 13:33:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 13:33:29.459972 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b88c231-2f32-4de5-9230-f623fa53abdb None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:29.547409 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70851588-43ce-4855-b025-8c5afdee8a9d None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:29.550503 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:29.579481 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b88c231-2f32-4de5-9230-f623fa53abdb None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:29.586293 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8983b95-be65-4584-b401-849eb5945cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:29.586921 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8983b95-be65-4584-b401-849eb5945cb7 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:29.587202 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8983b95-be65-4584-b401-849eb5945cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:29.587472 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8983b95-be65-4584-b401-849eb5945cb7 None admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:29.588710 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8983b95-be65-4584-b401-849eb5945cb7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:29.589011 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8983b95-be65-4584-b401-849eb5945cb7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:29.590409 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8983b95-be65-4584-b401-849eb5945cb7 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:29.602483 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 258/522] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Jul 26 13:33:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:29.603235 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8983b95-be65-4584-b401-849eb5945cb7 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:29.621891 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9641a8e8-636a-482d-b011-0516946a8231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:29.622572 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9641a8e8-636a-482d-b011-0516946a8231 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:29.622853 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9641a8e8-636a-482d-b011-0516946a8231 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:29.623128 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9641a8e8-636a-482d-b011-0516946a8231 None admin] PATH_INFO: `/v3/users` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:29.623933 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9641a8e8-636a-482d-b011-0516946a8231 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:29.624241 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9641a8e8-636a-482d-b011-0516946a8231 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:29.625701 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9641a8e8-636a-482d-b011-0516946a8231 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:29.645287 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 265/523] 10.222.0.58 () {56 vars in 1139 bytes} [Tue Jul 26 13:33:29 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:29.645287 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9641a8e8-636a-482d-b011-0516946a8231 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:29.687169 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:29.722453 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8983b95-be65-4584-b401-849eb5945cb7 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:29.748606 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9641a8e8-636a-482d-b011-0516946a8231 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:29.765106 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9224a1ae-21bb-489b-ac5c-fff2e04daf86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:29.765945 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9224a1ae-21bb-489b-ac5c-fff2e04daf86 None admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:29.766099 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9224a1ae-21bb-489b-ac5c-fff2e04daf86 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:29.766287 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9224a1ae-21bb-489b-ac5c-fff2e04daf86 None admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:29.767084 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9224a1ae-21bb-489b-ac5c-fff2e04daf86 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:29.768808 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9224a1ae-21bb-489b-ac5c-fff2e04daf86 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:29.793191 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 259/524] 10.222.0.58 () {60 vars in 1132 bytes} [Tue Jul 26 13:33:29 2022] POST /identity/v3/users => generated 247 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:33:29.794304 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9224a1ae-21bb-489b-ac5c-fff2e04daf86 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:29.801821 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d13323-6b28-4952-9ad1-8ee041cd3893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:29.801821 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d13323-6b28-4952-9ad1-8ee041cd3893 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:29.802190 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d13323-6b28-4952-9ad1-8ee041cd3893 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:29.802271 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d13323-6b28-4952-9ad1-8ee041cd3893 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:29.811497 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d13323-6b28-4952-9ad1-8ee041cd3893 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:29.813272 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d13323-6b28-4952-9ad1-8ee041cd3893 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:29.815959 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:29.830326 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-16d13323-6b28-4952-9ad1-8ee041cd3893 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 13:33:29.830326 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:29.830326 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:29.830326 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:29.830326 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:29.830326 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:29.830326 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:29.830326 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:29.830326 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:29.830326 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:29.830326 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:29.830326 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:29.830326 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 13:33:29.830326 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 13:33:29.830326 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 13:33:29.831316 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 13:33:29.831316 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:29.831316 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:29.831316 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:29.831316 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:29.831316 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:29.831316 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:29.831316 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:29.831316 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:29.831316 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:29.831316 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:29.831316 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:29.831316 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:29.831316 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:29.831316 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:29.831316 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:29.831316 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:29.832206 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 13:33:29.832206 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 13:33:29.832206 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 13:33:29.832206 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 13:33:29.832206 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 13:33:29.832206 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 13:33:29.832206 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 13:33:29.832206 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:33:29.832206 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 266/525] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Jul 26 13:33:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:29.833279 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d13323-6b28-4952-9ad1-8ee041cd3893 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:29.933118 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9224a1ae-21bb-489b-ac5c-fff2e04daf86 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:29.965593 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d13323-6b28-4952-9ad1-8ee041cd3893 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:29.968666 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:29.978928 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64404bb9-a859-448d-86a7-216248b6dcf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:29.979614 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64404bb9-a859-448d-86a7-216248b6dcf1 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:29.980130 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64404bb9-a859-448d-86a7-216248b6dcf1 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:29.980406 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64404bb9-a859-448d-86a7-216248b6dcf1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:29.991940 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64404bb9-a859-448d-86a7-216248b6dcf1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:29.993421 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64404bb9-a859-448d-86a7-216248b6dcf1 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:30.014635 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be41eee-59c7-4476-b15b-c65bde1cac8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:30.014872 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-64404bb9-a859-448d-86a7-216248b6dcf1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:30.014872 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:30.014872 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:30.014872 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:30.014872 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:30.014872 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:30.014872 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:30.014872 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:30.014872 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:30.014872 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:30.014872 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:30.014872 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:30.014872 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 13:33:30.014872 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 13:33:30.014872 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 13:33:30.016123 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 13:33:30.016123 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:30.016123 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:30.016123 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:30.016123 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:30.016123 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:30.016123 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:30.016123 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:30.016123 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:30.016123 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:30.016123 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:30.016123 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:30.016123 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:30.016123 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:30.016123 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:30.016123 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:30.016123 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:30.017163 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 13:33:30.017163 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 13:33:30.017163 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:30.017163 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:33:30.017432 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be41eee-59c7-4476-b15b-c65bde1cac8c None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:30.017432 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be41eee-59c7-4476-b15b-c65bde1cac8c None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:30.017432 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be41eee-59c7-4476-b15b-c65bde1cac8c None admin] PATH_INFO: `/v3/projects/service` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:30.017649 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 260/526] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:33:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:30.017649 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64404bb9-a859-448d-86a7-216248b6dcf1 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:30.026978 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be41eee-59c7-4476-b15b-c65bde1cac8c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:30.028529 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be41eee-59c7-4476-b15b-c65bde1cac8c None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:30.041273 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-7be41eee-59c7-4476-b15b-c65bde1cac8c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 13:33:30.041273 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:30.041273 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:30.041273 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:30.041273 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:30.041273 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:30.041273 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:30.041273 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:30.041273 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:30.041273 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:30.041273 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:30.041273 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:30.041273 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 13:33:30.041273 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 13:33:30.041273 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 13:33:30.042250 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 13:33:30.042250 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:30.042250 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:30.042250 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:30.042250 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:30.042250 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:30.042250 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:30.042250 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:30.042250 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:30.042250 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:30.042250 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:30.042250 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:30.042250 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:30.042250 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:30.042250 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:30.042250 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:30.042250 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:30.043147 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 13:33:30.043147 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 13:33:30.043147 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 13:33:30.043147 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 13:33:30.043147 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 13:33:30.043147 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 13:33:30.043147 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 13:33:30.043147 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:33:30.043147 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 267/527] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Jul 26 13:33:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:30.043620 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be41eee-59c7-4476-b15b-c65bde1cac8c None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:30.127206 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:30.150132 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64404bb9-a859-448d-86a7-216248b6dcf1 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:30.178324 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be41eee-59c7-4476-b15b-c65bde1cac8c None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:30.190351 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913511a8-7e5d-4d39-b2b4-a1b08ca15b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:30.190879 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913511a8-7e5d-4d39-b2b4-a1b08ca15b69 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:30.191474 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913511a8-7e5d-4d39-b2b4-a1b08ca15b69 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:30.191756 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913511a8-7e5d-4d39-b2b4-a1b08ca15b69 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:30.229747 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe38ebbc-31d7-4a0c-a98c-1c92aec4ef7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:30.230463 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe38ebbc-31d7-4a0c-a98c-1c92aec4ef7a None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:30.230742 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe38ebbc-31d7-4a0c-a98c-1c92aec4ef7a None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:30.231012 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe38ebbc-31d7-4a0c-a98c-1c92aec4ef7a None admin] PATH_INFO: `/v3/projects` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:30.231766 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe38ebbc-31d7-4a0c-a98c-1c92aec4ef7a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:30.232076 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe38ebbc-31d7-4a0c-a98c-1c92aec4ef7a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:30.233529 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe38ebbc-31d7-4a0c-a98c-1c92aec4ef7a None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:30.259692 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 268/528] 10.222.0.58 () {56 vars in 1148 bytes} [Tue Jul 26 13:33:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:30.259692 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe38ebbc-31d7-4a0c-a98c-1c92aec4ef7a None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:30.275277 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:30.298025 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913511a8-7e5d-4d39-b2b4-a1b08ca15b69 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:30.299938 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913511a8-7e5d-4d39-b2b4-a1b08ca15b69 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:30.370355 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe38ebbc-31d7-4a0c-a98c-1c92aec4ef7a None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:30.392914 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:30.412967 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94cb8d6-f03d-47f7-bec5-d0cca6f0e359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:30.414466 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94cb8d6-f03d-47f7-bec5-d0cca6f0e359 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:30.414888 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94cb8d6-f03d-47f7-bec5-d0cca6f0e359 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:30.415240 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94cb8d6-f03d-47f7-bec5-d0cca6f0e359 None admin] PATH_INFO: `/v3/domains` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:30.416519 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94cb8d6-f03d-47f7-bec5-d0cca6f0e359 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:30.416899 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94cb8d6-f03d-47f7-bec5-d0cca6f0e359 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:30.418462 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94cb8d6-f03d-47f7-bec5-d0cca6f0e359 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:30.432916 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 261/529] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Jul 26 13:33:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 13:33:30.432916 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913511a8-7e5d-4d39-b2b4-a1b08ca15b69 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:30.433636 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 269/530] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Jul 26 13:33:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:30.434522 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94cb8d6-f03d-47f7-bec5-d0cca6f0e359 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:30.549151 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:30.565287 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94cb8d6-f03d-47f7-bec5-d0cca6f0e359 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:30.569652 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913511a8-7e5d-4d39-b2b4-a1b08ca15b69 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:30.605255 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2df8b8-9515-46e9-a81d-baed076e0e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:30.609028 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2df8b8-9515-46e9-a81d-baed076e0e3e None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:30.609028 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2df8b8-9515-46e9-a81d-baed076e0e3e None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:30.609028 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2df8b8-9515-46e9-a81d-baed076e0e3e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:30.609028 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2df8b8-9515-46e9-a81d-baed076e0e3e None admin] RBAC: Adding query filter params (role.id=17e0f3c73a8a4a54b3e7b78878dfbf09, scope.project.id=1e8bb4385c204210a1bc77b1f0d654f3, user.id=c69161cd520b4b29bf50f3bf2f1c033b) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:30.609028 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2df8b8-9515-46e9-a81d-baed076e0e3e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:30.609449 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2df8b8-9515-46e9-a81d-baed076e0e3e None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:30.626122 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 270/531] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Jul 26 13:33:30 2022] GET /identity/v3/role_assignments?user.id=c69161cd520b4b29bf50f3bf2f1c033b&scope.project.id=1e8bb4385c204210a1bc77b1f0d654f3&role.id=17e0f3c73a8a4a54b3e7b78878dfbf09 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:30.626122 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2df8b8-9515-46e9-a81d-baed076e0e3e None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:30.631806 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b577598-8a42-4e06-8a7e-d4506fb85a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:30.632594 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b577598-8a42-4e06-8a7e-d4506fb85a2c None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:30.632979 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b577598-8a42-4e06-8a7e-d4506fb85a2c None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:30.633357 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b577598-8a42-4e06-8a7e-d4506fb85a2c None admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:30.634497 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b577598-8a42-4e06-8a7e-d4506fb85a2c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:30.634929 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b577598-8a42-4e06-8a7e-d4506fb85a2c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:30.636553 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b577598-8a42-4e06-8a7e-d4506fb85a2c None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:30.661718 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 262/532] 10.222.0.58 () {56 vars in 1148 bytes} [Tue Jul 26 13:33:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:30.662864 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b577598-8a42-4e06-8a7e-d4506fb85a2c None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:30.701549 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:30.795162 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b577598-8a42-4e06-8a7e-d4506fb85a2c None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:30.798302 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2df8b8-9515-46e9-a81d-baed076e0e3e None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:30.845150 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ee8c6b-c753-4fb4-86f8-99a73323d108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:30.850830 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ee8c6b-c753-4fb4-86f8-99a73323d108 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:30.850830 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ee8c6b-c753-4fb4-86f8-99a73323d108 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:30.850830 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ee8c6b-c753-4fb4-86f8-99a73323d108 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:30.856751 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704ee8af-802c-48af-b8b9-556fb46807db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:30.857420 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704ee8af-802c-48af-b8b9-556fb46807db None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:30.857727 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704ee8af-802c-48af-b8b9-556fb46807db None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:30.858001 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704ee8af-802c-48af-b8b9-556fb46807db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:30.863003 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ee8c6b-c753-4fb4-86f8-99a73323d108 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:30.864804 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ee8c6b-c753-4fb4-86f8-99a73323d108 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:30.872335 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704ee8af-802c-48af-b8b9-556fb46807db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:30.874257 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704ee8af-802c-48af-b8b9-556fb46807db None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:30.887866 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-92ee8c6b-c753-4fb4-86f8-99a73323d108 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 13:33:30.887866 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:30.887866 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:30.887866 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:30.887866 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:30.887866 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:30.887866 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:30.887866 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:30.887866 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:30.887866 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:30.887866 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:30.887866 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:30.887866 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 13:33:30.887866 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 13:33:30.887866 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 13:33:30.888787 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 13:33:30.888787 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:30.888787 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:30.888787 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:30.888787 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:30.888787 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:30.888787 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:30.888787 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:30.888787 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:30.888787 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:30.888787 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:30.888787 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:30.888787 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:30.888787 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:30.888787 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:30.888787 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:30.888787 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:30.889656 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 13:33:30.889656 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 13:33:30.889656 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 13:33:30.889656 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 13:33:30.889656 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 13:33:30.889656 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 13:33:30.889656 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 13:33:30.889656 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:33:30.890522 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-704ee8af-802c-48af-b8b9-556fb46807db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:30.890522 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:30.890522 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:30.890522 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:30.890522 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:30.890522 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:30.890522 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:30.890522 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:30.890522 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:30.890522 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:30.890522 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:30.890522 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:30.890522 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 13:33:30.890522 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 13:33:30.890522 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 13:33:30.891309 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 13:33:30.891309 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:30.891309 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:30.891309 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:30.891309 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:30.891309 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:30.891309 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:30.891309 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:30.891309 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:30.891309 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:30.891309 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:30.891309 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:30.891309 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:30.891309 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:30.891309 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:30.891309 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:30.891309 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:30.892167 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 13:33:30.892167 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 13:33:30.892167 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:30.892167 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:33:30.895422 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 263/533] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Jul 26 13:33:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:30.897152 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ee8c6b-c753-4fb4-86f8-99a73323d108 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:30.897844 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 271/534] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:33:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:30.901630 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:30.903566 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704ee8af-802c-48af-b8b9-556fb46807db None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:31.042648 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:31.054396 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704ee8af-802c-48af-b8b9-556fb46807db None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:31.063136 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ee8c6b-c753-4fb4-86f8-99a73323d108 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:31.101834 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de61c89a-ac0d-4b7f-a77c-57eaebf027d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:31.103137 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de61c89a-ac0d-4b7f-a77c-57eaebf027d6 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:31.103659 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de61c89a-ac0d-4b7f-a77c-57eaebf027d6 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:31.104138 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de61c89a-ac0d-4b7f-a77c-57eaebf027d6 None admin] PATH_INFO: `/v3/projects` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:31.105833 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de61c89a-ac0d-4b7f-a77c-57eaebf027d6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:31.106165 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de61c89a-ac0d-4b7f-a77c-57eaebf027d6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:31.107626 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de61c89a-ac0d-4b7f-a77c-57eaebf027d6 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:31.121735 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c005ebd-4248-42b7-81d5-9ead2ea9a676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:31.122741 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c005ebd-4248-42b7-81d5-9ead2ea9a676 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:31.123233 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c005ebd-4248-42b7-81d5-9ead2ea9a676 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:31.123696 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c005ebd-4248-42b7-81d5-9ead2ea9a676 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:31.124698 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c005ebd-4248-42b7-81d5-9ead2ea9a676 None admin] RBAC: Adding query filter params (role.id=17e0f3c73a8a4a54b3e7b78878dfbf09, scope.project.id=1e8bb4385c204210a1bc77b1f0d654f3, user.id=72e400f267fd488d8439426a81ba0a25) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:31.125255 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c005ebd-4248-42b7-81d5-9ead2ea9a676 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:31.127009 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c005ebd-4248-42b7-81d5-9ead2ea9a676 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:31.128840 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 272/535] 10.222.0.58 () {56 vars in 1148 bytes} [Tue Jul 26 13:33:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:31.129832 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de61c89a-ac0d-4b7f-a77c-57eaebf027d6 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:31.140724 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 264/536] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Jul 26 13:33:30 2022] GET /identity/v3/role_assignments?user.id=72e400f267fd488d8439426a81ba0a25&scope.project.id=1e8bb4385c204210a1bc77b1f0d654f3&role.id=17e0f3c73a8a4a54b3e7b78878dfbf09 => generated 259 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:31.142097 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c005ebd-4248-42b7-81d5-9ead2ea9a676 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:31.247892 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de61c89a-ac0d-4b7f-a77c-57eaebf027d6 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:31.269857 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:31.301576 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0dc01e-01a5-4aaa-aa88-1c0d09e76242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:31.302281 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0dc01e-01a5-4aaa-aa88-1c0d09e76242 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:31.302569 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0dc01e-01a5-4aaa-aa88-1c0d09e76242 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:31.302846 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0dc01e-01a5-4aaa-aa88-1c0d09e76242 None admin] PATH_INFO: `/v3/domains` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:31.304364 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0dc01e-01a5-4aaa-aa88-1c0d09e76242 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:31.304677 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0dc01e-01a5-4aaa-aa88-1c0d09e76242 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:31.306131 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0dc01e-01a5-4aaa-aa88-1c0d09e76242 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:31.338323 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 273/537] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Jul 26 13:33:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:31.340253 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0dc01e-01a5-4aaa-aa88-1c0d09e76242 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:31.368709 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:31.372616 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c005ebd-4248-42b7-81d5-9ead2ea9a676 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:31.441446 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efe498d-bed6-463a-b25e-77d21e3c6c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:31.442189 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efe498d-bed6-463a-b25e-77d21e3c6c72 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:31.442503 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efe498d-bed6-463a-b25e-77d21e3c6c72 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:31.442760 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efe498d-bed6-463a-b25e-77d21e3c6c72 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:31.443569 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efe498d-bed6-463a-b25e-77d21e3c6c72 None admin] RBAC: Adding query filter params (role.id=17e0f3c73a8a4a54b3e7b78878dfbf09, scope.project.id=1e8bb4385c204210a1bc77b1f0d654f3, user.id=eaf065aef9a046b2adaec34816a336b7) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:31.443883 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efe498d-bed6-463a-b25e-77d21e3c6c72 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:31.445336 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efe498d-bed6-463a-b25e-77d21e3c6c72 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:31.455772 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 265/538] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Jul 26 13:33:31 2022] GET /identity/v3/role_assignments?user.id=eaf065aef9a046b2adaec34816a336b7&scope.project.id=1e8bb4385c204210a1bc77b1f0d654f3&role.id=17e0f3c73a8a4a54b3e7b78878dfbf09 => generated 259 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:31.505749 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0dc01e-01a5-4aaa-aa88-1c0d09e76242 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:31.566944 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e369f3-eb6d-4f60-9307-1b7e6611600c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:31.568023 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e369f3-eb6d-4f60-9307-1b7e6611600c None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:31.568504 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e369f3-eb6d-4f60-9307-1b7e6611600c None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:31.568971 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e369f3-eb6d-4f60-9307-1b7e6611600c None admin] PATH_INFO: `/v3/projects/service` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:31.584035 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e369f3-eb6d-4f60-9307-1b7e6611600c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:31.585896 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e369f3-eb6d-4f60-9307-1b7e6611600c None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:31.604870 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-64e369f3-eb6d-4f60-9307-1b7e6611600c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 13:33:31.604870 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:31.604870 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:31.604870 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:31.604870 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:31.604870 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:31.604870 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:31.604870 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:31.604870 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:31.604870 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:31.604870 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:31.604870 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:31.604870 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 13:33:31.604870 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 13:33:31.604870 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 13:33:31.605813 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 13:33:31.605813 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:31.605813 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:31.605813 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:31.605813 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:31.605813 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:31.605813 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:31.605813 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:31.605813 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:31.605813 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:31.605813 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:31.605813 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:31.605813 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:31.605813 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:31.605813 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:31.605813 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:31.605813 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:31.606653 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 13:33:31.606653 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 13:33:31.606653 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 13:33:31.606653 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 13:33:31.606653 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 13:33:31.606653 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 13:33:31.606653 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 13:33:31.606653 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:33:31.608886 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 274/539] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Jul 26 13:33:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 13:33:31.614910 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efe498d-bed6-463a-b25e-77d21e3c6c72 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:31.791673 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efe498d-bed6-463a-b25e-77d21e3c6c72 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:31.838877 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:31.846945 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96767baf-5634-4848-86f2-749f05fc408a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:31.847588 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96767baf-5634-4848-86f2-749f05fc408a None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:31.847875 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96767baf-5634-4848-86f2-749f05fc408a None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:31.848153 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96767baf-5634-4848-86f2-749f05fc408a None admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:31.848919 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96767baf-5634-4848-86f2-749f05fc408a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:31.849232 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96767baf-5634-4848-86f2-749f05fc408a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:31.850706 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96767baf-5634-4848-86f2-749f05fc408a None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:31.871520 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 266/540] 10.222.0.58 () {56 vars in 1148 bytes} [Tue Jul 26 13:33:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:31.877153 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e369f3-eb6d-4f60-9307-1b7e6611600c None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:31.976284 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:32.013602 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e369f3-eb6d-4f60-9307-1b7e6611600c None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:32.087010 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf39e43c-166d-4032-bd58-d0a998646f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:32.087705 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf39e43c-166d-4032-bd58-d0a998646f6a None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:32.088006 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf39e43c-166d-4032-bd58-d0a998646f6a None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:32.088290 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf39e43c-166d-4032-bd58-d0a998646f6a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:32.089097 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf39e43c-166d-4032-bd58-d0a998646f6a None admin] RBAC: Adding query filter params (role.id=17e0f3c73a8a4a54b3e7b78878dfbf09, scope.project.id=1e8bb4385c204210a1bc77b1f0d654f3, user.id=3223a78866d94c2ba66402911c2dfae9) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:32.089413 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf39e43c-166d-4032-bd58-d0a998646f6a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:32.095353 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf39e43c-166d-4032-bd58-d0a998646f6a None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:32.101714 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 275/541] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Jul 26 13:33:31 2022] GET /identity/v3/role_assignments?user.id=3223a78866d94c2ba66402911c2dfae9&scope.project.id=1e8bb4385c204210a1bc77b1f0d654f3&role.id=17e0f3c73a8a4a54b3e7b78878dfbf09 => generated 259 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:32.325109 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313e7ba2-ef56-4709-99c8-e975414deac6 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:32.325292 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313e7ba2-ef56-4709-99c8-e975414deac6 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:32.325601 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313e7ba2-ef56-4709-99c8-e975414deac6 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:32.327307 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 267/542] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:33:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:33:32.334031 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a7de78-e6f5-44e5-a85e-856ca95162d5 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:32.334528 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a7de78-e6f5-44e5-a85e-856ca95162d5 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:32.335007 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a7de78-e6f5-44e5-a85e-856ca95162d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:32.363992 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-f4a7de78-e6f5-44e5-a85e-856ca95162d5 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:33:32.365957 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-f4a7de78-e6f5-44e5-a85e-856ca95162d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:33:32.472221 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 276/543] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:33:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 13:33:32.515414 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313e7ba2-ef56-4709-99c8-e975414deac6 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:32.662176 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313e7ba2-ef56-4709-99c8-e975414deac6 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:32.712597 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ea0d7c-2688-49a0-bb44-a258a8e0a033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:32.714254 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ea0d7c-2688-49a0-bb44-a258a8e0a033 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:32.714530 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ea0d7c-2688-49a0-bb44-a258a8e0a033 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:32.715537 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ea0d7c-2688-49a0-bb44-a258a8e0a033 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:32.731961 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ea0d7c-2688-49a0-bb44-a258a8e0a033 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:33:32.736989 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-d7ea0d7c-2688-49a0-bb44-a258a8e0a033 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 13:33:32.736989 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:32.736989 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:32.736989 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:32.736989 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:32.736989 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:32.736989 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:32.736989 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:32.736989 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:32.736989 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:32.736989 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:32.736989 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:32.736989 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:33:32.736989 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:33:32.736989 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:33:32.736989 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:33:32.738653 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:33:32.738653 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:33:32.738653 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:33:32.738653 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:33:32.738653 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:32.738653 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:32.738653 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:32.738653 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:32.738653 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:32.738653 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:32.738653 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:32.738653 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:32.738653 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:32.738653 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:32.738653 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:32.738653 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:32.740229 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:32.740229 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:32.740229 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:32.740229 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:32.740229 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:33:32.740229 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:33:32.740229 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:33:32.740229 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:33:32.740229 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:33:32.740229 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:33:32.740229 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 13:33:32.740229 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:33:32.740229 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 268/544] 10.222.0.58 () {56 vars in 1102 bytes} [Tue Jul 26 13:33:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:32.746643 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a7de78-e6f5-44e5-a85e-856ca95162d5 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:32.858009 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:32.891706 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a7de78-e6f5-44e5-a85e-856ca95162d5 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:32.930843 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb2c1a5-eab9-4274-990d-87fa28364d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:32.932289 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb2c1a5-eab9-4274-990d-87fa28364d68 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:32.932723 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb2c1a5-eab9-4274-990d-87fa28364d68 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:32.933166 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb2c1a5-eab9-4274-990d-87fa28364d68 None admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:32.934519 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb2c1a5-eab9-4274-990d-87fa28364d68 None admin] RBAC: Adding query filter params (name=service) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:32.934881 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb2c1a5-eab9-4274-990d-87fa28364d68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:32.936343 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb2c1a5-eab9-4274-990d-87fa28364d68 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:32.953029 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 277/545] 10.222.0.58 () {56 vars in 1103 bytes} [Tue Jul 26 13:33:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:32.958374 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ea0d7c-2688-49a0-bb44-a258a8e0a033 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:33.028930 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:33.047582 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048adf41-f9af-4a35-b8b4-735282fab24a None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:33.047910 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048adf41-f9af-4a35-b8b4-735282fab24a None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:33.048134 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048adf41-f9af-4a35-b8b4-735282fab24a None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:33.049977 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 278/546] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:33:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 13:33:33.060169 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd209c9-cce2-4902-9a4a-9180f23f783e None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:33.060477 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd209c9-cce2-4902-9a4a-9180f23f783e None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:33.060721 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd209c9-cce2-4902-9a4a-9180f23f783e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:33.086710 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-9bd209c9-cce2-4902-9a4a-9180f23f783e None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:33:33.088465 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-9bd209c9-cce2-4902-9a4a-9180f23f783e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:33:33.127381 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ea0d7c-2688-49a0-bb44-a258a8e0a033 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:33.202167 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d73864-f52e-4a2b-8c70-139d5a2d7251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:33.202779 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d73864-f52e-4a2b-8c70-139d5a2d7251 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:33.203110 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d73864-f52e-4a2b-8c70-139d5a2d7251 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:33.203386 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d73864-f52e-4a2b-8c70-139d5a2d7251 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:33.224418 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d73864-f52e-4a2b-8c70-139d5a2d7251 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:33.227750 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d73864-f52e-4a2b-8c70-139d5a2d7251 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:33.228803 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 279/547] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:33:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 13:33:33.249446 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd209c9-cce2-4902-9a4a-9180f23f783e None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:33.257874 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-96d73864-f52e-4a2b-8c70-139d5a2d7251 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:33.257874 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:33.257874 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:33.257874 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:33.257874 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:33.257874 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:33.257874 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:33.257874 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:33.257874 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:33.257874 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:33.257874 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:33.257874 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:33.257874 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 13:33:33.257874 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 13:33:33.257874 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 13:33:33.258778 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 13:33:33.258778 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:33.258778 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:33.258778 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:33.258778 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:33.258778 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:33.258778 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:33.258778 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:33.258778 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:33.258778 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:33.258778 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:33.258778 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:33.258778 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:33.258778 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:33.258778 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:33.258778 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:33.258778 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:33.259650 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 13:33:33.259650 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 13:33:33.259650 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:33.259650 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:33:33.262468 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 269/548] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:33:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:33.266871 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d73864-f52e-4a2b-8c70-139d5a2d7251 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:33.399629 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d73864-f52e-4a2b-8c70-139d5a2d7251 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:33.401867 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:33.402531 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:33.449498 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f432bd9-0912-4f42-a2fb-285158f413ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:33.450133 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f432bd9-0912-4f42-a2fb-285158f413ad None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:33.450414 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f432bd9-0912-4f42-a2fb-285158f413ad None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:33.450684 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f432bd9-0912-4f42-a2fb-285158f413ad None admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:33.451384 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f432bd9-0912-4f42-a2fb-285158f413ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:33.451690 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f432bd9-0912-4f42-a2fb-285158f413ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:33.453122 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f432bd9-0912-4f42-a2fb-285158f413ad None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:33.470926 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 270/549] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Jul 26 13:33:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:33.481392 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f432bd9-0912-4f42-a2fb-285158f413ad None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:33.504289 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd209c9-cce2-4902-9a4a-9180f23f783e None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:33.571783 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949afa36-7f14-45b7-b462-eb1ef938e5dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:33.572979 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949afa36-7f14-45b7-b462-eb1ef938e5dc None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:33.573261 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949afa36-7f14-45b7-b462-eb1ef938e5dc None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:33.573559 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949afa36-7f14-45b7-b462-eb1ef938e5dc None admin] PATH_INFO: `/v3/roles/service` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:33.597227 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949afa36-7f14-45b7-b462-eb1ef938e5dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:33:33.602287 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-949afa36-7f14-45b7-b462-eb1ef938e5dc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 13:33:33.602287 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:33.602287 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:33.602287 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:33.602287 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:33.602287 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:33.602287 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:33.602287 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:33.602287 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:33.602287 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:33.602287 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:33.602287 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:33.602287 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:33:33.602287 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:33:33.602287 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:33:33.602287 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:33:33.603691 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:33:33.603691 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:33:33.603691 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:33:33.603691 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:33:33.603691 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:33.603691 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:33.603691 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:33.603691 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:33.603691 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:33.603691 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:33.603691 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:33.603691 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:33.603691 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:33.603691 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:33.603691 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:33.603691 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:33.604648 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:33.604648 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:33.604648 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:33.604648 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:33.604648 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:33:33.604648 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:33:33.604648 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:33:33.604648 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:33:33.604648 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:33:33.604648 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:33:33.604648 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 13:33:33.604648 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:33:33.604648 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 280/550] 10.222.0.58 () {56 vars in 1102 bytes} [Tue Jul 26 13:33:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 355 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 13:33:33.612229 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949afa36-7f14-45b7-b462-eb1ef938e5dc None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:33.681884 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f432bd9-0912-4f42-a2fb-285158f413ad None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:33.720412 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6219d459-e393-498c-b973-39af19483409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:33.721194 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6219d459-e393-498c-b973-39af19483409 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:33.721606 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6219d459-e393-498c-b973-39af19483409 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:33.721997 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6219d459-e393-498c-b973-39af19483409 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:33.730330 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6219d459-e393-498c-b973-39af19483409 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:33.732068 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6219d459-e393-498c-b973-39af19483409 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:33.746040 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-6219d459-e393-498c-b973-39af19483409 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 26 13:33:33.746040 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:33.746040 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:33.746040 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:33.746040 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:33.746040 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:33.746040 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:33.746040 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:33.746040 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:33.746040 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:33.746040 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:33.746040 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:33.746040 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 13:33:33.746040 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 13:33:33.746040 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 13:33:33.746040 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 13:33:33.747229 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:33.747229 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:33.747229 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 13:33:33.747229 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 13:33:33.747229 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 13:33:33.747229 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 13:33:33.747229 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:33.747229 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:33.747229 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:33.747229 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:33.747229 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:33.747229 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:33.747229 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:33.747229 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:33.747229 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:33.747229 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:33.747229 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:33.747229 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:33.748499 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:33.748499 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:33.748499 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 13:33:33.748499 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 13:33:33.748499 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 13:33:33.748499 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 13:33:33.748499 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 13:33:33.748499 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 13:33:33.748499 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 26 13:33:33.748499 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:33:33.754715 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 271/551] 10.222.0.58 () {56 vars in 1096 bytes} [Tue Jul 26 13:33:33 2022] GET /identity/v3/users/swift => generated 83 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 13:33:33.756100 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6219d459-e393-498c-b973-39af19483409 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:33.784781 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:33.838517 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949afa36-7f14-45b7-b462-eb1ef938e5dc None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:33.885430 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:33.914984 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6698c2-84ce-4cbc-986a-b782904f2013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:33.915951 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6698c2-84ce-4cbc-986a-b782904f2013 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:33.916556 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6698c2-84ce-4cbc-986a-b782904f2013 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:33.917454 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6698c2-84ce-4cbc-986a-b782904f2013 None admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:33.918761 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6698c2-84ce-4cbc-986a-b782904f2013 None admin] RBAC: Adding query filter params (name=service) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:33.919302 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6698c2-84ce-4cbc-986a-b782904f2013 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:33.921854 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6698c2-84ce-4cbc-986a-b782904f2013 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:33.928257 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6219d459-e393-498c-b973-39af19483409 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:33.933724 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 281/552] 10.222.0.58 () {56 vars in 1103 bytes} [Tue Jul 26 13:33:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:33.935408 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545c73d1-26b1-4fd6-8a55-453dcba79871 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:33.935966 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545c73d1-26b1-4fd6-8a55-453dcba79871 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:33.936503 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545c73d1-26b1-4fd6-8a55-453dcba79871 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:33.939419 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 282/553] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:33:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:33:33.940680 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545c73d1-26b1-4fd6-8a55-453dcba79871 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:33.979519 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab48d948-abd2-4e97-bcbe-3e784b539c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:33.980042 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab48d948-abd2-4e97-bcbe-3e784b539c21 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:33.980334 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab48d948-abd2-4e97-bcbe-3e784b539c21 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:33.980608 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab48d948-abd2-4e97-bcbe-3e784b539c21 None admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:33.981625 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab48d948-abd2-4e97-bcbe-3e784b539c21 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:33.981998 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab48d948-abd2-4e97-bcbe-3e784b539c21 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:33.983444 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab48d948-abd2-4e97-bcbe-3e784b539c21 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:34.014561 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 272/554] 10.222.0.58 () {56 vars in 1135 bytes} [Tue Jul 26 13:33:33 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:33:34.016220 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920d82b0-8ebe-4dba-872e-31b51acd658c None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:34.016450 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920d82b0-8ebe-4dba-872e-31b51acd658c None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:34.016726 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920d82b0-8ebe-4dba-872e-31b51acd658c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:34.043482 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-920d82b0-8ebe-4dba-872e-31b51acd658c None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:33:34.044706 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-920d82b0-8ebe-4dba-872e-31b51acd658c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:33:34.099976 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545c73d1-26b1-4fd6-8a55-453dcba79871 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:34.125533 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 273/555] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:33:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 13:33:34.125533 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920d82b0-8ebe-4dba-872e-31b51acd658c None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:34.144564 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:34.148544 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0b9207-bebb-485b-aed9-186929f98c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:34.149309 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0b9207-bebb-485b-aed9-186929f98c45 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:34.149612 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0b9207-bebb-485b-aed9-186929f98c45 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:34.149888 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0b9207-bebb-485b-aed9-186929f98c45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:34.161331 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0b9207-bebb-485b-aed9-186929f98c45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:34.162837 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0b9207-bebb-485b-aed9-186929f98c45 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:34.187545 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-3a0b9207-bebb-485b-aed9-186929f98c45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:34.187545 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:34.187545 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:34.187545 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:34.187545 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:34.187545 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:34.187545 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:34.187545 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:34.187545 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:34.187545 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:34.187545 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:34.187545 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:34.187545 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 13:33:34.187545 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 13:33:34.187545 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 13:33:34.188591 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 13:33:34.188591 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:34.188591 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:34.188591 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:34.188591 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:34.188591 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:34.188591 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:34.188591 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:34.188591 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:34.188591 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:34.188591 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:34.188591 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:34.188591 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:34.188591 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:34.188591 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:34.188591 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:34.188591 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:34.189621 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 13:33:34.189621 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 13:33:34.189621 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:34.189621 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:33:34.193381 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 283/556] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:33:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:34.194747 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19279129-d5d7-4cdb-9404-e701dc269d78 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:34.195036 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19279129-d5d7-4cdb-9404-e701dc269d78 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:34.195300 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19279129-d5d7-4cdb-9404-e701dc269d78 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:34.196613 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 284/557] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:33:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:33:34.198941 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19279129-d5d7-4cdb-9404-e701dc269d78 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:34.249426 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:34.264752 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920d82b0-8ebe-4dba-872e-31b51acd658c None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:34.309739 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00550362-4814-452f-945b-65e1f18e1c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:34.310365 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00550362-4814-452f-945b-65e1f18e1c01 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:34.310638 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00550362-4814-452f-945b-65e1f18e1c01 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:34.310909 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00550362-4814-452f-945b-65e1f18e1c01 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:34.332221 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19279129-d5d7-4cdb-9404-e701dc269d78 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:34.380957 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1476e19d-27a4-4341-9b51-eaf6596848f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:34.381648 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1476e19d-27a4-4341-9b51-eaf6596848f3 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:34.381930 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1476e19d-27a4-4341-9b51-eaf6596848f3 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:34.382202 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1476e19d-27a4-4341-9b51-eaf6596848f3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:34.392953 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1476e19d-27a4-4341-9b51-eaf6596848f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:33:34.397720 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00550362-4814-452f-945b-65e1f18e1c01 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:34.398985 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00550362-4814-452f-945b-65e1f18e1c01 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:34.399096 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-1476e19d-27a4-4341-9b51-eaf6596848f3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 13:33:34.399096 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:34.399096 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:34.399096 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:34.399096 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:34.399096 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:34.399096 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:34.399096 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:34.399096 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:34.399096 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:34.399096 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:34.399096 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:34.399096 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:33:34.399096 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:33:34.399096 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:33:34.399096 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:33:34.400407 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:33:34.400407 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:33:34.400407 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:33:34.400407 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:33:34.400407 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:34.400407 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:34.400407 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:34.400407 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:34.400407 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:34.400407 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:34.400407 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:34.400407 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:34.400407 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:34.400407 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:34.400407 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:34.400407 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:34.401766 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:34.401766 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:34.401766 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:34.401766 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:34.401766 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:33:34.401766 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:33:34.401766 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:33:34.401766 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:33:34.401766 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:33:34.401766 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:33:34.401766 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 13:33:34.401766 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:33:34.405231 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 285/558] 10.222.0.58 () {56 vars in 1102 bytes} [Tue Jul 26 13:33:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:34.406197 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1476e19d-27a4-4341-9b51-eaf6596848f3 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:34.414951 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:34.557917 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:34.564974 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 274/559] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Jul 26 13:33:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 13:33:34.564974 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f11b01-df91-42d1-96d5-b8a914559387 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:34.564974 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f11b01-df91-42d1-96d5-b8a914559387 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:34.564974 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f11b01-df91-42d1-96d5-b8a914559387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:34.574764 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1476e19d-27a4-4341-9b51-eaf6596848f3 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:34.581839 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-15f11b01-df91-42d1-96d5-b8a914559387 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:33:34.583062 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-15f11b01-df91-42d1-96d5-b8a914559387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:33:34.616209 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7cb722-dad6-4882-90f6-82f96ab1cc01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:34.616836 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7cb722-dad6-4882-90f6-82f96ab1cc01 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:34.617127 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7cb722-dad6-4882-90f6-82f96ab1cc01 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:34.617406 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7cb722-dad6-4882-90f6-82f96ab1cc01 None admin] PATH_INFO: `/v3/domains` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:34.618151 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7cb722-dad6-4882-90f6-82f96ab1cc01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:34.618460 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7cb722-dad6-4882-90f6-82f96ab1cc01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:34.619919 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7cb722-dad6-4882-90f6-82f96ab1cc01 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:34.632199 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 286/560] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Jul 26 13:33:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:34.633054 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7cb722-dad6-4882-90f6-82f96ab1cc01 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:34.658746 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 275/561] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:33:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 13:33:34.659610 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f11b01-df91-42d1-96d5-b8a914559387 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:34.704397 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:34.747704 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7cb722-dad6-4882-90f6-82f96ab1cc01 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:34.771749 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f11b01-df91-42d1-96d5-b8a914559387 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:34.788121 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fbf00f-e532-40a4-aa60-06c990cb1fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:34.789130 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fbf00f-e532-40a4-aa60-06c990cb1fd4 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:34.789657 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fbf00f-e532-40a4-aa60-06c990cb1fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:34.789943 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fbf00f-e532-40a4-aa60-06c990cb1fd4 None admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:34.791335 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fbf00f-e532-40a4-aa60-06c990cb1fd4 None admin] RBAC: Adding query filter params (name=service) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:34.791641 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fbf00f-e532-40a4-aa60-06c990cb1fd4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:34.793009 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fbf00f-e532-40a4-aa60-06c990cb1fd4 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:34.801713 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 287/562] 10.222.0.58 () {56 vars in 1103 bytes} [Tue Jul 26 13:33:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:34.803043 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2454035-37f4-4724-a29c-bbc91f8680fd None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:34.803327 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2454035-37f4-4724-a29c-bbc91f8680fd None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:34.803600 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2454035-37f4-4724-a29c-bbc91f8680fd None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:34.809023 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 288/563] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:33:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:33:34.809023 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2454035-37f4-4724-a29c-bbc91f8680fd None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:34.813901 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72a1c24-8544-4a03-ac37-e7a8b367db0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:34.814555 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72a1c24-8544-4a03-ac37-e7a8b367db0d None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:34.814834 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72a1c24-8544-4a03-ac37-e7a8b367db0d None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:34.815102 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72a1c24-8544-4a03-ac37-e7a8b367db0d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:34.826594 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72a1c24-8544-4a03-ac37-e7a8b367db0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:34.827796 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:34.828053 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72a1c24-8544-4a03-ac37-e7a8b367db0d None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:34.843762 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-a72a1c24-8544-4a03-ac37-e7a8b367db0d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:34.843762 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:34.843762 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:34.843762 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:34.843762 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:34.843762 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:34.843762 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:34.843762 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:34.843762 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:34.843762 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:34.843762 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:34.843762 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:34.843762 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 13:33:34.843762 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 13:33:34.843762 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 13:33:34.844749 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 13:33:34.844749 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:34.844749 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:34.844749 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:34.844749 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:34.844749 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:34.844749 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:34.844749 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:34.844749 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:34.844749 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:34.844749 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:34.844749 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:34.844749 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:34.844749 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:34.844749 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:34.844749 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:34.844749 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:34.845670 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 13:33:34.845670 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 13:33:34.845670 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:34.845670 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:33:34.845670 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 276/564] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:33:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:34.845938 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72a1c24-8544-4a03-ac37-e7a8b367db0d None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:34.925302 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2454035-37f4-4724-a29c-bbc91f8680fd None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:34.942996 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:34.953462 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72a1c24-8544-4a03-ac37-e7a8b367db0d None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:34.975331 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eca4e65-9e09-4393-ab1e-1128ad3c6927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:34.975937 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eca4e65-9e09-4393-ab1e-1128ad3c6927 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:34.976223 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eca4e65-9e09-4393-ab1e-1128ad3c6927 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:34.976501 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eca4e65-9e09-4393-ab1e-1128ad3c6927 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:34.984785 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eca4e65-9e09-4393-ab1e-1128ad3c6927 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:34.986281 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eca4e65-9e09-4393-ab1e-1128ad3c6927 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:34.988335 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79e1330-34ad-4207-87be-10d35f1b9896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:34.988981 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79e1330-34ad-4207-87be-10d35f1b9896 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:34.989262 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79e1330-34ad-4207-87be-10d35f1b9896 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:34.989565 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79e1330-34ad-4207-87be-10d35f1b9896 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:34.998768 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-4eca4e65-9e09-4393-ab1e-1128ad3c6927 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 26 13:33:34.998768 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:34.998768 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:34.998768 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:34.998768 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:34.998768 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:34.998768 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:34.998768 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:34.998768 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:34.998768 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:34.998768 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:34.998768 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:34.998768 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 13:33:34.998768 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 13:33:34.998768 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 13:33:34.998768 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 13:33:34.999944 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:34.999944 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:34.999944 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 13:33:34.999944 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 13:33:34.999944 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 13:33:34.999944 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 13:33:34.999944 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:34.999944 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:34.999944 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:34.999944 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:34.999944 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:34.999944 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:34.999944 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:34.999944 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:34.999944 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:34.999944 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:34.999944 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:34.999944 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:35.000894 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:35.000894 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:35.000894 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 13:33:35.000894 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 13:33:35.000894 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 13:33:35.000894 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 13:33:35.000894 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 13:33:35.000894 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 13:33:35.000894 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 26 13:33:35.000894 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:33:35.000894 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 289/565] 10.222.0.58 () {56 vars in 1093 bytes} [Tue Jul 26 13:33:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:35.000894 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eca4e65-9e09-4393-ab1e-1128ad3c6927 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:35.002614 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79e1330-34ad-4207-87be-10d35f1b9896 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:33:35.008064 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-e79e1330-34ad-4207-87be-10d35f1b9896 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 13:33:35.008064 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:35.008064 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:35.008064 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:35.008064 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:35.008064 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:35.008064 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:35.008064 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:35.008064 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:35.008064 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:35.008064 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:35.008064 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:35.008064 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:33:35.008064 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:33:35.008064 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:33:35.008064 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:33:35.009033 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:33:35.009033 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:33:35.009033 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:33:35.009033 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:33:35.009033 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:35.009033 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:35.009033 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:35.009033 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:35.009033 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:35.009033 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:35.009033 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:35.009033 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:35.009033 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:35.009033 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:35.009033 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:35.009033 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:35.010479 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:35.010479 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:35.010479 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:35.010479 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:35.010479 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:33:35.010479 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:33:35.010479 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:33:35.010479 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:33:35.010479 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:33:35.010479 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:33:35.010479 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 13:33:35.010479 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:33:35.010479 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 277/566] 10.222.0.58 () {56 vars in 1102 bytes} [Tue Jul 26 13:33:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:35.011177 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff83613-ec42-4a49-8921-9226f427327d None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:35.011269 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff83613-ec42-4a49-8921-9226f427327d None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:35.011660 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff83613-ec42-4a49-8921-9226f427327d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:35.033740 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-4ff83613-ec42-4a49-8921-9226f427327d None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:33:35.034963 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-4ff83613-ec42-4a49-8921-9226f427327d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:33:35.088447 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:35.106407 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 278/567] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:33:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 13:33:35.107412 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff83613-ec42-4a49-8921-9226f427327d None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:35.111496 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eca4e65-9e09-4393-ab1e-1128ad3c6927 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:35.149450 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5903c0b-abb5-4a12-b96d-6e05465c4429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:35.150709 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5903c0b-abb5-4a12-b96d-6e05465c4429 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:35.150997 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5903c0b-abb5-4a12-b96d-6e05465c4429 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:35.151274 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5903c0b-abb5-4a12-b96d-6e05465c4429 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:35.162121 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5903c0b-abb5-4a12-b96d-6e05465c4429 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:35.165935 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5903c0b-abb5-4a12-b96d-6e05465c4429 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:35.178692 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-b5903c0b-abb5-4a12-b96d-6e05465c4429 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:35.178692 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:35.178692 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:35.178692 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:35.178692 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:35.178692 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:35.178692 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:35.178692 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:35.178692 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:35.178692 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:35.178692 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:35.178692 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:35.178692 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 13:33:35.178692 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 13:33:35.178692 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 13:33:35.179621 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 13:33:35.179621 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:35.179621 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:35.179621 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:35.179621 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:35.179621 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:35.179621 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:35.179621 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:35.179621 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:35.179621 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:35.179621 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:35.179621 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:35.179621 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:35.179621 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:35.179621 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:35.179621 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:35.179621 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:35.180491 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 13:33:35.180491 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 13:33:35.180491 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:35.180491 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:33:35.180491 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 290/568] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:33:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:35.180768 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5903c0b-abb5-4a12-b96d-6e05465c4429 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:35.213650 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:35.233032 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff83613-ec42-4a49-8921-9226f427327d None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:35.271739 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe03969-3d0e-4d9e-8ef9-79995c5d918a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:35.272354 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe03969-3d0e-4d9e-8ef9-79995c5d918a None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:35.272632 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe03969-3d0e-4d9e-8ef9-79995c5d918a None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:35.272902 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe03969-3d0e-4d9e-8ef9-79995c5d918a None admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:35.273723 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe03969-3d0e-4d9e-8ef9-79995c5d918a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:35.274003 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe03969-3d0e-4d9e-8ef9-79995c5d918a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:35.275406 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe03969-3d0e-4d9e-8ef9-79995c5d918a None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:35.286943 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 279/569] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Jul 26 13:33:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:35.287784 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe03969-3d0e-4d9e-8ef9-79995c5d918a None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:35.292864 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5903c0b-abb5-4a12-b96d-6e05465c4429 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:35.320269 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:35.329438 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09609b56-efec-4557-80f1-01dfb4796320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:35.330100 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09609b56-efec-4557-80f1-01dfb4796320 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:35.330381 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09609b56-efec-4557-80f1-01dfb4796320 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:35.330657 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09609b56-efec-4557-80f1-01dfb4796320 None admin] PATH_INFO: `/v3/users` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:35.331459 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09609b56-efec-4557-80f1-01dfb4796320 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:35.331769 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09609b56-efec-4557-80f1-01dfb4796320 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:35.333211 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09609b56-efec-4557-80f1-01dfb4796320 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:35.351010 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 291/570] 10.222.0.58 () {56 vars in 1133 bytes} [Tue Jul 26 13:33:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:35.351783 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09609b56-efec-4557-80f1-01dfb4796320 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:35.393872 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe03969-3d0e-4d9e-8ef9-79995c5d918a None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:35.428096 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158360cc-a2ab-41d1-9198-cde50764787f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:35.428722 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158360cc-a2ab-41d1-9198-cde50764787f None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:35.429005 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158360cc-a2ab-41d1-9198-cde50764787f None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:35.429281 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158360cc-a2ab-41d1-9198-cde50764787f None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:35.430033 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158360cc-a2ab-41d1-9198-cde50764787f None admin] RBAC: Adding query filter params (name=service) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:35.430352 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158360cc-a2ab-41d1-9198-cde50764787f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:35.432303 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158360cc-a2ab-41d1-9198-cde50764787f None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:35.444998 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 280/571] 10.222.0.58 () {56 vars in 1103 bytes} [Tue Jul 26 13:33:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:35.445894 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158360cc-a2ab-41d1-9198-cde50764787f None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:35.450267 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:35.462955 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09609b56-efec-4557-80f1-01dfb4796320 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:35.500201 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a043253b-7ed8-4e1d-b923-7598d9bd4f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:35.500853 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a043253b-7ed8-4e1d-b923-7598d9bd4f67 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:35.501130 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a043253b-7ed8-4e1d-b923-7598d9bd4f67 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:35.501407 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a043253b-7ed8-4e1d-b923-7598d9bd4f67 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:35.514340 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a043253b-7ed8-4e1d-b923-7598d9bd4f67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:33:35.519257 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-a043253b-7ed8-4e1d-b923-7598d9bd4f67 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 13:33:35.519257 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:35.519257 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:35.519257 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:35.519257 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:35.519257 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:35.519257 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:35.519257 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:35.519257 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:35.519257 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:35.519257 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:35.519257 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:35.519257 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:33:35.519257 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:33:35.519257 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:33:35.519257 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:33:35.520406 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:33:35.520406 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:33:35.520406 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:33:35.520406 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:33:35.520406 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:35.520406 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:35.520406 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:35.520406 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:35.520406 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:35.520406 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:35.520406 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:35.520406 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:35.520406 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:35.520406 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:35.520406 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:35.520406 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:35.521428 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:35.521428 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:35.521428 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:35.521428 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:35.521428 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:33:35.521428 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:33:35.521428 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:33:35.521428 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:33:35.521428 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:33:35.521428 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:33:35.521428 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 13:33:35.521428 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:33:35.521428 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 292/572] 10.222.0.58 () {56 vars in 1102 bytes} [Tue Jul 26 13:33:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:35.522269 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a043253b-7ed8-4e1d-b923-7598d9bd4f67 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:35.552346 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158360cc-a2ab-41d1-9198-cde50764787f None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:35.570222 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:35.590897 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4707da35-2878-40ee-a707-cbbce445c006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:35.591514 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4707da35-2878-40ee-a707-cbbce445c006 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:35.591792 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4707da35-2878-40ee-a707-cbbce445c006 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:35.592061 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4707da35-2878-40ee-a707-cbbce445c006 None admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:35.592745 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4707da35-2878-40ee-a707-cbbce445c006 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:35.593051 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4707da35-2878-40ee-a707-cbbce445c006 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:35.594447 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4707da35-2878-40ee-a707-cbbce445c006 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:35.607620 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 281/573] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Jul 26 13:33:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:35.608388 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4707da35-2878-40ee-a707-cbbce445c006 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:35.633029 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a043253b-7ed8-4e1d-b923-7598d9bd4f67 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:35.669444 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24c9703-2e47-48ff-9b28-34e4d722645b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:35.670130 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24c9703-2e47-48ff-9b28-34e4d722645b None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:35.670413 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24c9703-2e47-48ff-9b28-34e4d722645b None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:35.670686 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24c9703-2e47-48ff-9b28-34e4d722645b None admin] PATH_INFO: `/v3/projects/service` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:35.676309 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:35.678399 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24c9703-2e47-48ff-9b28-34e4d722645b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:35.680021 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24c9703-2e47-48ff-9b28-34e4d722645b None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:35.691646 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-d24c9703-2e47-48ff-9b28-34e4d722645b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 13:33:35.691646 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:35.691646 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:35.691646 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:35.691646 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:35.691646 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:35.691646 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:35.691646 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:35.691646 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:35.691646 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:35.691646 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:35.691646 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:35.691646 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 13:33:35.691646 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 13:33:35.691646 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 13:33:35.692625 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 13:33:35.692625 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:35.692625 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:35.692625 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:35.692625 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:35.692625 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:35.692625 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:35.692625 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:35.692625 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:35.692625 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:35.692625 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:35.692625 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:35.692625 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:35.692625 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:35.692625 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:35.692625 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:35.692625 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:35.693607 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 13:33:35.693607 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 13:33:35.693607 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 13:33:35.693607 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 13:33:35.693607 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 13:33:35.693607 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 13:33:35.693607 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 13:33:35.693607 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:33:35.693607 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 293/574] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Jul 26 13:33:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:35.693607 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24c9703-2e47-48ff-9b28-34e4d722645b None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:35.719543 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4707da35-2878-40ee-a707-cbbce445c006 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:35.754671 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a09662-5e31-4ebc-b4ee-6b1b3381d3af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:35.755326 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a09662-5e31-4ebc-b4ee-6b1b3381d3af None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:35.755602 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a09662-5e31-4ebc-b4ee-6b1b3381d3af None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:35.755868 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a09662-5e31-4ebc-b4ee-6b1b3381d3af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:35.767390 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a09662-5e31-4ebc-b4ee-6b1b3381d3af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:35.770002 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a09662-5e31-4ebc-b4ee-6b1b3381d3af None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:35.790189 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-87a09662-5e31-4ebc-b4ee-6b1b3381d3af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:35.790189 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:35.790189 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:35.790189 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:35.790189 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:35.790189 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:35.790189 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:35.790189 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:35.790189 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:35.790189 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:35.790189 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:35.790189 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:35.790189 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 13:33:35.790189 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 13:33:35.790189 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 13:33:35.791291 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 13:33:35.791291 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:35.791291 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:35.791291 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:35.791291 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:35.791291 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:35.791291 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:35.791291 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:35.791291 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:35.791291 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:35.791291 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:35.791291 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:35.791291 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:35.791291 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:35.791291 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:35.791291 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:35.791291 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:35.792230 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 13:33:35.792230 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 13:33:35.792230 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:35.792230 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:33:35.792931 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 282/575] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:33:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:35.793976 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a09662-5e31-4ebc-b4ee-6b1b3381d3af None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:35.811114 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24c9703-2e47-48ff-9b28-34e4d722645b None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:35.818317 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:35.853902 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de7bb1c-37e9-452f-a465-5b57b78caed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:35.854557 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de7bb1c-37e9-452f-a465-5b57b78caed4 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:35.854845 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de7bb1c-37e9-452f-a465-5b57b78caed4 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:35.855194 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de7bb1c-37e9-452f-a465-5b57b78caed4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:35.866259 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de7bb1c-37e9-452f-a465-5b57b78caed4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:35.867652 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de7bb1c-37e9-452f-a465-5b57b78caed4 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:35.882988 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-7de7bb1c-37e9-452f-a465-5b57b78caed4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:35.882988 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:35.882988 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:35.882988 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:35.882988 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:35.882988 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:35.882988 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:35.882988 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:35.882988 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:35.882988 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:35.882988 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:35.882988 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:35.882988 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 13:33:35.882988 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 13:33:35.882988 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 13:33:35.884123 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 13:33:35.884123 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:35.884123 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:35.884123 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:35.884123 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:35.884123 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:35.884123 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:35.884123 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:35.884123 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:35.884123 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:35.884123 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:35.884123 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:35.884123 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:35.884123 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:35.884123 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:35.884123 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:35.884123 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:35.885618 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 13:33:35.885618 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 13:33:35.885618 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:35.885618 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:33:35.885618 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 294/576] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:33:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:35.885618 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de7bb1c-37e9-452f-a465-5b57b78caed4 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:35.902030 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a09662-5e31-4ebc-b4ee-6b1b3381d3af None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:35.922795 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:35.944808 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c765ae1-065e-4a83-8301-fb308cf99aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:35.945604 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c765ae1-065e-4a83-8301-fb308cf99aaf None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:35.945892 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c765ae1-065e-4a83-8301-fb308cf99aaf None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:35.946166 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c765ae1-065e-4a83-8301-fb308cf99aaf None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:35.947476 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c765ae1-065e-4a83-8301-fb308cf99aaf None admin] RBAC: Adding query filter params (name=service) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:35.947781 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c765ae1-065e-4a83-8301-fb308cf99aaf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:35.949150 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c765ae1-065e-4a83-8301-fb308cf99aaf None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:35.957891 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 283/577] 10.222.0.58 () {56 vars in 1103 bytes} [Tue Jul 26 13:33:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:35.958990 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c765ae1-065e-4a83-8301-fb308cf99aaf None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:35.999334 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de7bb1c-37e9-452f-a465-5b57b78caed4 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:36.034357 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e3f721-aff6-4e94-800d-9f477e444c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:36.035310 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e3f721-aff6-4e94-800d-9f477e444c18 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:36.035742 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e3f721-aff6-4e94-800d-9f477e444c18 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:36.036163 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e3f721-aff6-4e94-800d-9f477e444c18 None admin] PATH_INFO: `/v3/users/glance` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:36.047136 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e3f721-aff6-4e94-800d-9f477e444c18 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:36.048834 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e3f721-aff6-4e94-800d-9f477e444c18 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:36.061342 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-e5e3f721-aff6-4e94-800d-9f477e444c18 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 26 13:33:36.061342 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:36.061342 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:36.061342 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:36.061342 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:36.061342 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:36.061342 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:36.061342 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:36.061342 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:36.061342 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:36.061342 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:36.061342 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:36.061342 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 13:33:36.061342 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 13:33:36.061342 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 13:33:36.061342 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 13:33:36.062413 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:36.062413 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:36.062413 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 13:33:36.062413 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 13:33:36.062413 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 13:33:36.062413 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 13:33:36.062413 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:36.062413 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:36.062413 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:36.062413 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:36.062413 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:36.062413 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:36.062413 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:36.062413 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:36.062413 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:36.062413 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:36.062413 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:36.062413 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:36.063352 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:36.063352 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:36.063352 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 13:33:36.063352 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 13:33:36.063352 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 13:33:36.063352 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 13:33:36.063352 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 13:33:36.063352 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 13:33:36.063352 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 26 13:33:36.063352 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:33:36.063352 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 295/578] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Jul 26 13:33:35 2022] GET /identity/v3/users/glance => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:36.063931 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c765ae1-065e-4a83-8301-fb308cf99aaf None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:36.064342 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e3f721-aff6-4e94-800d-9f477e444c18 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:36.084659 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:36.106489 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25db03d7-b8c4-47f3-a4d4-27021f20b0e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:36.107111 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25db03d7-b8c4-47f3-a4d4-27021f20b0e6 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:36.107390 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25db03d7-b8c4-47f3-a4d4-27021f20b0e6 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:36.107664 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25db03d7-b8c4-47f3-a4d4-27021f20b0e6 None admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:36.108438 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25db03d7-b8c4-47f3-a4d4-27021f20b0e6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:36.108742 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25db03d7-b8c4-47f3-a4d4-27021f20b0e6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:36.110219 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25db03d7-b8c4-47f3-a4d4-27021f20b0e6 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:36.123301 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 284/579] 10.222.0.58 () {56 vars in 1148 bytes} [Tue Jul 26 13:33:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:36.124059 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25db03d7-b8c4-47f3-a4d4-27021f20b0e6 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:36.163468 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:36.179032 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e3f721-aff6-4e94-800d-9f477e444c18 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:36.213877 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56d9cbf-f9c4-4673-a471-d5c8a850cb62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:36.214508 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56d9cbf-f9c4-4673-a471-d5c8a850cb62 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:36.214792 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56d9cbf-f9c4-4673-a471-d5c8a850cb62 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:36.215073 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56d9cbf-f9c4-4673-a471-d5c8a850cb62 None admin] PATH_INFO: `/v3/domains` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:36.215768 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56d9cbf-f9c4-4673-a471-d5c8a850cb62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:36.216076 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56d9cbf-f9c4-4673-a471-d5c8a850cb62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:36.217468 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56d9cbf-f9c4-4673-a471-d5c8a850cb62 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:36.224770 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25db03d7-b8c4-47f3-a4d4-27021f20b0e6 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:36.229808 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 296/580] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Jul 26 13:33:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:36.230659 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56d9cbf-f9c4-4673-a471-d5c8a850cb62 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:36.263428 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10be3265-1c29-4a07-b93e-1a55fd9503f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:36.264051 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10be3265-1c29-4a07-b93e-1a55fd9503f9 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:36.264332 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10be3265-1c29-4a07-b93e-1a55fd9503f9 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:36.264597 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10be3265-1c29-4a07-b93e-1a55fd9503f9 None admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:36.271302 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10be3265-1c29-4a07-b93e-1a55fd9503f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:36.271302 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10be3265-1c29-4a07-b93e-1a55fd9503f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:36.271302 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10be3265-1c29-4a07-b93e-1a55fd9503f9 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:36.278772 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 285/581] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Jul 26 13:33:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:36.279611 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10be3265-1c29-4a07-b93e-1a55fd9503f9 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:36.320135 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:36.343228 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56d9cbf-f9c4-4673-a471-d5c8a850cb62 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:36.382430 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199f52a4-36a5-4a11-ade0-068d99f20816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:36.382990 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199f52a4-36a5-4a11-ade0-068d99f20816 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:36.383267 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199f52a4-36a5-4a11-ade0-068d99f20816 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:36.383539 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199f52a4-36a5-4a11-ade0-068d99f20816 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:36.387846 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:36.392025 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10be3265-1c29-4a07-b93e-1a55fd9503f9 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:36.395350 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199f52a4-36a5-4a11-ade0-068d99f20816 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:36.397055 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199f52a4-36a5-4a11-ade0-068d99f20816 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:36.412720 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-199f52a4-36a5-4a11-ade0-068d99f20816 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:36.412720 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:36.412720 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:36.412720 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:36.412720 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:36.412720 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:36.412720 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:36.412720 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:36.412720 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:36.412720 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:36.412720 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:36.412720 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:36.412720 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 13:33:36.412720 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 13:33:36.412720 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 13:33:36.413913 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 13:33:36.413913 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:36.413913 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:36.413913 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:36.413913 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:36.413913 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:36.413913 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:36.413913 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:36.413913 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:36.413913 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:36.413913 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:36.413913 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:36.413913 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:36.413913 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:36.413913 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:36.413913 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:36.413913 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:36.420113 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 13:33:36.420113 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 13:33:36.420113 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:36.420113 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:33:36.420113 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 297/582] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:33:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:36.420113 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199f52a4-36a5-4a11-ade0-068d99f20816 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:36.432704 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097fd434-136d-488f-b41f-3a8f6e56be8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:36.433697 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097fd434-136d-488f-b41f-3a8f6e56be8e None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:36.433987 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097fd434-136d-488f-b41f-3a8f6e56be8e None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:36.434262 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097fd434-136d-488f-b41f-3a8f6e56be8e None admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:36.435067 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097fd434-136d-488f-b41f-3a8f6e56be8e None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:36.435377 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097fd434-136d-488f-b41f-3a8f6e56be8e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:36.436928 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097fd434-136d-488f-b41f-3a8f6e56be8e None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:36.459009 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 286/583] 10.222.0.58 () {56 vars in 1137 bytes} [Tue Jul 26 13:33:36 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:36.459880 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097fd434-136d-488f-b41f-3a8f6e56be8e None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:36.533375 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199f52a4-36a5-4a11-ade0-068d99f20816 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:36.565335 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:36.573064 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5da2cc6-ab1e-4894-ba4d-f1c59cba515c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:36.573272 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097fd434-136d-488f-b41f-3a8f6e56be8e None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:36.573827 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5da2cc6-ab1e-4894-ba4d-f1c59cba515c None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:36.574164 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5da2cc6-ab1e-4894-ba4d-f1c59cba515c None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:36.574494 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5da2cc6-ab1e-4894-ba4d-f1c59cba515c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:36.575422 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5da2cc6-ab1e-4894-ba4d-f1c59cba515c None admin] RBAC: Adding query filter params (role.id=17e0f3c73a8a4a54b3e7b78878dfbf09, scope.project.id=1e8bb4385c204210a1bc77b1f0d654f3, user.id=3ab2819bac844163ae4305e4f2186605) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:36.575851 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5da2cc6-ab1e-4894-ba4d-f1c59cba515c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:36.577375 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5da2cc6-ab1e-4894-ba4d-f1c59cba515c None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:36.591902 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 298/584] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Jul 26 13:33:36 2022] GET /identity/v3/role_assignments?user.id=3ab2819bac844163ae4305e4f2186605&scope.project.id=1e8bb4385c204210a1bc77b1f0d654f3&role.id=17e0f3c73a8a4a54b3e7b78878dfbf09 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:36.592739 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5da2cc6-ab1e-4894-ba4d-f1c59cba515c None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:36.617411 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc33d907-b8a7-4d44-93ae-3e9f4376e21c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:36.618089 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc33d907-b8a7-4d44-93ae-3e9f4376e21c None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:36.618370 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc33d907-b8a7-4d44-93ae-3e9f4376e21c None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:36.618708 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc33d907-b8a7-4d44-93ae-3e9f4376e21c None admin] PATH_INFO: `/v3/projects/service` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:36.626873 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc33d907-b8a7-4d44-93ae-3e9f4376e21c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:36.628368 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc33d907-b8a7-4d44-93ae-3e9f4376e21c None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:36.640891 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-dc33d907-b8a7-4d44-93ae-3e9f4376e21c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 13:33:36.640891 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:36.640891 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:36.640891 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:36.640891 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:36.640891 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:36.640891 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:36.640891 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:36.640891 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:36.640891 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:36.640891 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:36.640891 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:36.640891 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 13:33:36.640891 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 13:33:36.640891 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 13:33:36.641896 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 13:33:36.641896 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:36.641896 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:36.641896 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:36.641896 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:36.641896 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:36.641896 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:36.641896 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:36.641896 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:36.641896 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:36.641896 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:36.641896 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:36.641896 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:36.641896 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:36.641896 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:36.641896 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:36.641896 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:36.648027 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 13:33:36.648027 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 13:33:36.648027 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 13:33:36.648027 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 13:33:36.648027 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 13:33:36.648027 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 13:33:36.648027 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 13:33:36.648027 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:33:36.648027 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 287/585] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Jul 26 13:33:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:36.648027 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc33d907-b8a7-4d44-93ae-3e9f4376e21c None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:36.668295 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:36.744467 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5da2cc6-ab1e-4894-ba4d-f1c59cba515c None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:36.778757 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc33d907-b8a7-4d44-93ae-3e9f4376e21c None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:36.785085 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202d5985-9700-4812-b048-c90f2fe2123f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:36.786048 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202d5985-9700-4812-b048-c90f2fe2123f None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:36.786564 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202d5985-9700-4812-b048-c90f2fe2123f None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:36.786858 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202d5985-9700-4812-b048-c90f2fe2123f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:36.795684 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202d5985-9700-4812-b048-c90f2fe2123f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:36.797130 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202d5985-9700-4812-b048-c90f2fe2123f None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:36.813565 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-202d5985-9700-4812-b048-c90f2fe2123f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 26 13:33:36.813565 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:36.813565 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:36.813565 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:36.813565 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:36.813565 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:36.813565 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:36.813565 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:36.813565 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:36.813565 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:36.813565 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:36.813565 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:36.813565 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 13:33:36.813565 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 13:33:36.813565 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 13:33:36.813565 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 13:33:36.814546 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:36.814546 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:36.814546 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 13:33:36.814546 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 13:33:36.814546 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 13:33:36.814546 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 13:33:36.814546 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:36.814546 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:36.814546 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:36.814546 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:36.814546 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:36.814546 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:36.814546 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:36.814546 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:36.814546 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:36.814546 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:36.814546 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:36.814546 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:36.815461 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:36.815461 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:36.815461 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 13:33:36.815461 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 13:33:36.815461 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 13:33:36.815461 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 13:33:36.815461 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 13:33:36.815461 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 13:33:36.815461 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 26 13:33:36.815461 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:33:36.819499 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 299/586] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Jul 26 13:33:36 2022] GET /identity/v3/users/cinder => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:36.820426 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202d5985-9700-4812-b048-c90f2fe2123f None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:36.825299 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99954560-4082-4241-870a-de1ab697df83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:36.825962 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99954560-4082-4241-870a-de1ab697df83 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:36.826245 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99954560-4082-4241-870a-de1ab697df83 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:36.826519 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99954560-4082-4241-870a-de1ab697df83 None admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:36.827261 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99954560-4082-4241-870a-de1ab697df83 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:36.827569 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99954560-4082-4241-870a-de1ab697df83 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:36.829122 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99954560-4082-4241-870a-de1ab697df83 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:36.844538 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 288/587] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Jul 26 13:33:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:36.845427 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99954560-4082-4241-870a-de1ab697df83 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:36.889283 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:36.937279 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:36.945719 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202d5985-9700-4812-b048-c90f2fe2123f None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:36.965623 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99954560-4082-4241-870a-de1ab697df83 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:36.995715 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d351a29d-161f-493a-bcad-74e11f6c8c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:36.996740 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d351a29d-161f-493a-bcad-74e11f6c8c9d None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:36.997255 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d351a29d-161f-493a-bcad-74e11f6c8c9d None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:36.997886 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d351a29d-161f-493a-bcad-74e11f6c8c9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:37.012668 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e901bee2-533f-47d6-9c57-4c0bd307e03c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:37.013297 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e901bee2-533f-47d6-9c57-4c0bd307e03c None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:37.013606 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e901bee2-533f-47d6-9c57-4c0bd307e03c None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:37.013710 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d351a29d-161f-493a-bcad-74e11f6c8c9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:37.013868 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e901bee2-533f-47d6-9c57-4c0bd307e03c None admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:37.014624 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e901bee2-533f-47d6-9c57-4c0bd307e03c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:37.014928 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e901bee2-533f-47d6-9c57-4c0bd307e03c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:37.015913 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d351a29d-161f-493a-bcad-74e11f6c8c9d None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:37.016373 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e901bee2-533f-47d6-9c57-4c0bd307e03c None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:37.034482 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 289/588] 10.222.0.58 () {56 vars in 1148 bytes} [Tue Jul 26 13:33:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:37.034482 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e901bee2-533f-47d6-9c57-4c0bd307e03c None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:37.034770 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-d351a29d-161f-493a-bcad-74e11f6c8c9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:37.034770 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:37.034770 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:37.034770 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:37.034770 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:37.034770 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:37.034770 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:37.034770 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:37.034770 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:37.034770 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:37.034770 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:37.034770 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:37.034770 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 13:33:37.034770 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 13:33:37.034770 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 13:33:37.035548 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 13:33:37.035548 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:37.035548 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:37.035548 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:37.035548 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:37.035548 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:37.035548 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:37.035548 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:37.035548 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:37.035548 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:37.035548 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:37.035548 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:37.035548 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:37.035548 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:37.035548 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:37.035548 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:37.035548 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:37.036494 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 13:33:37.036494 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 13:33:37.036494 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:37.036494 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:33:37.037816 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 300/589] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:33:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:37.038929 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d351a29d-161f-493a-bcad-74e11f6c8c9d None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:37.153772 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e901bee2-533f-47d6-9c57-4c0bd307e03c None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:37.160961 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d351a29d-161f-493a-bcad-74e11f6c8c9d None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:37.171271 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:37.196289 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ad296b-87c2-417f-a0c6-9af5564e8821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:37.196289 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ad296b-87c2-417f-a0c6-9af5564e8821 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:37.196289 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ad296b-87c2-417f-a0c6-9af5564e8821 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:37.196289 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ad296b-87c2-417f-a0c6-9af5564e8821 None admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:37.197292 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ad296b-87c2-417f-a0c6-9af5564e8821 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:37.197453 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ad296b-87c2-417f-a0c6-9af5564e8821 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:37.198910 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ad296b-87c2-417f-a0c6-9af5564e8821 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:37.202592 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:37.210803 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc8bbfe-b9cf-4159-b1da-aca23fa3adb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:37.212110 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc8bbfe-b9cf-4159-b1da-aca23fa3adb4 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:37.212461 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc8bbfe-b9cf-4159-b1da-aca23fa3adb4 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:37.213237 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc8bbfe-b9cf-4159-b1da-aca23fa3adb4 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:37.221719 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc8bbfe-b9cf-4159-b1da-aca23fa3adb4 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:37.223174 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc8bbfe-b9cf-4159-b1da-aca23fa3adb4 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:37.229976 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 290/590] 10.222.0.58 () {56 vars in 1137 bytes} [Tue Jul 26 13:33:37 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:37.229976 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ad296b-87c2-417f-a0c6-9af5564e8821 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:37.245231 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-acc8bbfe-b9cf-4159-b1da-aca23fa3adb4 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 26 13:33:37.245231 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:37.245231 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:37.245231 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:37.245231 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:37.245231 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:37.245231 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:37.245231 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:37.245231 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:37.245231 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:37.245231 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:37.245231 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:37.245231 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 13:33:37.245231 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 13:33:37.245231 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 13:33:37.245231 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 13:33:37.246383 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:37.246383 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:37.246383 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 13:33:37.246383 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 13:33:37.246383 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 13:33:37.246383 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 13:33:37.246383 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:37.246383 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:37.246383 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:37.246383 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:37.246383 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:37.246383 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:37.246383 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:37.246383 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:37.246383 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:37.246383 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:37.246383 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:37.246383 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:37.247358 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:37.247358 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:37.247358 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 13:33:37.247358 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 13:33:37.247358 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 13:33:37.247358 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 13:33:37.247358 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 13:33:37.247358 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 13:33:37.247358 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 26 13:33:37.247358 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:33:37.247358 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 301/591] 10.222.0.58 () {56 vars in 1102 bytes} [Tue Jul 26 13:33:37 2022] GET /identity/v3/users/neutron => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:37.247358 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc8bbfe-b9cf-4159-b1da-aca23fa3adb4 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:37.353017 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ad296b-87c2-417f-a0c6-9af5564e8821 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:37.386681 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc8bbfe-b9cf-4159-b1da-aca23fa3adb4 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:37.393118 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4a23a6-5d8f-41f9-9853-57669c1a1ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:37.393828 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4a23a6-5d8f-41f9-9853-57669c1a1ba6 None admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:37.394116 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4a23a6-5d8f-41f9-9853-57669c1a1ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:37.394397 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4a23a6-5d8f-41f9-9853-57669c1a1ba6 None admin] PATH_INFO: `/v3/projects/1e8bb4385c204210a1bc77b1f0d654f3/users/c69161cd520b4b29bf50f3bf2f1c033b/roles/17e0f3c73a8a4a54b3e7b78878dfbf09` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:37.412431 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4a23a6-5d8f-41f9-9853-57669c1a1ba6 None admin] RBAC: Authorizing `identity:create_grant(project_id=1e8bb4385c204210a1bc77b1f0d654f3, user_id=c69161cd520b4b29bf50f3bf2f1c033b, role_id=17e0f3c73a8a4a54b3e7b78878dfbf09)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:37.413912 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4a23a6-5d8f-41f9-9853-57669c1a1ba6 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:37.433675 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 291/592] 10.222.0.58 () {58 vars in 1439 bytes} [Tue Jul 26 13:33:37 2022] PUT /identity/v3/projects/1e8bb4385c204210a1bc77b1f0d654f3/users/c69161cd520b4b29bf50f3bf2f1c033b/roles/17e0f3c73a8a4a54b3e7b78878dfbf09 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:33:37.434856 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4a23a6-5d8f-41f9-9853-57669c1a1ba6 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:37.448249 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283ace8e-0ccf-4f89-a4b5-a92e11da7e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:37.449277 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283ace8e-0ccf-4f89-a4b5-a92e11da7e08 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:37.449850 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283ace8e-0ccf-4f89-a4b5-a92e11da7e08 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:37.450392 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283ace8e-0ccf-4f89-a4b5-a92e11da7e08 None admin] PATH_INFO: `/v3/domains` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:37.451973 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283ace8e-0ccf-4f89-a4b5-a92e11da7e08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:37.452459 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283ace8e-0ccf-4f89-a4b5-a92e11da7e08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:37.455084 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283ace8e-0ccf-4f89-a4b5-a92e11da7e08 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:37.484199 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 302/593] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Jul 26 13:33:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:37.484199 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283ace8e-0ccf-4f89-a4b5-a92e11da7e08 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:37.540144 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:37.540998 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:37.617753 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4a23a6-5d8f-41f9-9853-57669c1a1ba6 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:37.656879 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001ad683-ab03-4d93-a6a9-6eda4be2475a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:37.657609 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001ad683-ab03-4d93-a6a9-6eda4be2475a None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:37.657886 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001ad683-ab03-4d93-a6a9-6eda4be2475a None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:37.658166 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001ad683-ab03-4d93-a6a9-6eda4be2475a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:37.668809 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001ad683-ab03-4d93-a6a9-6eda4be2475a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:37.670346 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001ad683-ab03-4d93-a6a9-6eda4be2475a None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:37.670739 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283ace8e-0ccf-4f89-a4b5-a92e11da7e08 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:37.693898 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-001ad683-ab03-4d93-a6a9-6eda4be2475a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:37.693898 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:37.693898 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:37.693898 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:37.693898 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:37.693898 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:37.693898 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:37.693898 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:37.693898 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:37.693898 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:37.693898 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:37.693898 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:37.693898 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 13:33:37.693898 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 13:33:37.693898 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 13:33:37.695052 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 13:33:37.695052 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:37.695052 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:37.695052 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:37.695052 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:37.695052 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:37.695052 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:37.695052 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:37.695052 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:37.695052 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:37.695052 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:37.695052 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:37.695052 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:37.695052 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:37.695052 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:37.695052 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:37.695052 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:37.695962 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 13:33:37.695962 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 13:33:37.695962 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:37.695962 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:33:37.695962 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 292/594] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:33:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:37.696257 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001ad683-ab03-4d93-a6a9-6eda4be2475a None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:37.721629 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aa85a8-de0f-464d-a910-4cfa8768ed99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:37.722693 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aa85a8-de0f-464d-a910-4cfa8768ed99 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:37.723205 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aa85a8-de0f-464d-a910-4cfa8768ed99 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:37.723720 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aa85a8-de0f-464d-a910-4cfa8768ed99 None admin] PATH_INFO: `/v3/users` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:37.724758 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4aa85a8-de0f-464d-a910-4cfa8768ed99 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:37.725274 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4aa85a8-de0f-464d-a910-4cfa8768ed99 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:37.726959 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4aa85a8-de0f-464d-a910-4cfa8768ed99 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:37.751553 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 303/595] 10.222.0.58 () {56 vars in 1139 bytes} [Tue Jul 26 13:33:37 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:37.752784 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aa85a8-de0f-464d-a910-4cfa8768ed99 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:37.831103 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:37.838702 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001ad683-ab03-4d93-a6a9-6eda4be2475a None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:37.882895 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92758e27-111b-4527-a615-770a78742cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:37.883896 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92758e27-111b-4527-a615-770a78742cfc None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:37.884433 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:37.885072 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92758e27-111b-4527-a615-770a78742cfc None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:37.885590 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92758e27-111b-4527-a615-770a78742cfc None admin] PATH_INFO: `/v3/projects/service` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:37.899694 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92758e27-111b-4527-a615-770a78742cfc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:37.901816 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92758e27-111b-4527-a615-770a78742cfc None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:37.904533 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aa85a8-de0f-464d-a910-4cfa8768ed99 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:37.920527 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-92758e27-111b-4527-a615-770a78742cfc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 13:33:37.920527 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:37.920527 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:37.920527 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:37.920527 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:37.920527 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:37.920527 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:37.920527 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:37.920527 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:37.920527 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:37.920527 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:37.920527 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:37.920527 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 13:33:37.920527 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 13:33:37.920527 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 13:33:37.921939 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 13:33:37.921939 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:37.921939 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:37.921939 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:37.921939 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:37.921939 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:37.921939 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:37.921939 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:37.921939 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:37.921939 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:37.921939 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:37.921939 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:37.921939 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:37.921939 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:37.921939 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:37.921939 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:37.921939 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:37.922818 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 13:33:37.922818 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 13:33:37.922818 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 13:33:37.922818 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 13:33:37.922818 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 13:33:37.922818 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 13:33:37.922818 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 13:33:37.922818 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:33:37.925579 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 293/596] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Jul 26 13:33:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:37.926713 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92758e27-111b-4527-a615-770a78742cfc None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:37.953146 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6a97bf-1a54-4588-99c9-2e54060ca3a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:37.954189 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6a97bf-1a54-4588-99c9-2e54060ca3a5 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:37.954701 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6a97bf-1a54-4588-99c9-2e54060ca3a5 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:37.955173 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6a97bf-1a54-4588-99c9-2e54060ca3a5 None admin] PATH_INFO: `/v3/domains` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:37.956083 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6a97bf-1a54-4588-99c9-2e54060ca3a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:37.956593 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6a97bf-1a54-4588-99c9-2e54060ca3a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:37.958303 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6a97bf-1a54-4588-99c9-2e54060ca3a5 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:37.976994 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 304/597] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Jul 26 13:33:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:37.978709 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6a97bf-1a54-4588-99c9-2e54060ca3a5 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:38.094234 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92758e27-111b-4527-a615-770a78742cfc None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:38.106211 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6a97bf-1a54-4588-99c9-2e54060ca3a5 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:38.131263 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c67973-85e3-46cd-800c-fb0390d26c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:38.131929 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c67973-85e3-46cd-800c-fb0390d26c9b None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:38.132206 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c67973-85e3-46cd-800c-fb0390d26c9b None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:38.132578 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c67973-85e3-46cd-800c-fb0390d26c9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:38.144659 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c67973-85e3-46cd-800c-fb0390d26c9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:38.146278 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c67973-85e3-46cd-800c-fb0390d26c9b None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:38.147925 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddcbf10-4992-42ab-a21e-f91d8f15dd01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:38.148813 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddcbf10-4992-42ab-a21e-f91d8f15dd01 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:38.149299 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddcbf10-4992-42ab-a21e-f91d8f15dd01 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:38.149838 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddcbf10-4992-42ab-a21e-f91d8f15dd01 None admin] PATH_INFO: `/v3/projects` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:38.150913 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddcbf10-4992-42ab-a21e-f91d8f15dd01 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:38.151428 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddcbf10-4992-42ab-a21e-f91d8f15dd01 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:38.153054 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddcbf10-4992-42ab-a21e-f91d8f15dd01 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:38.161902 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-a1c67973-85e3-46cd-800c-fb0390d26c9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:38.161902 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:38.161902 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:38.161902 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:38.161902 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:38.161902 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:38.161902 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:38.161902 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:38.161902 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:38.161902 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:38.161902 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:38.161902 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:38.161902 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 13:33:38.161902 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 13:33:38.161902 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 13:33:38.163199 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 13:33:38.163199 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:38.163199 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:38.163199 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:38.163199 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:38.163199 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:38.163199 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:38.163199 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:38.163199 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:38.163199 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:38.163199 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:38.163199 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:38.163199 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:38.163199 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:38.163199 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:38.163199 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:38.163199 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:38.164066 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 13:33:38.164066 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 13:33:38.164066 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:38.164066 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:33:38.164066 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 294/598] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:33:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:38.164361 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c67973-85e3-46cd-800c-fb0390d26c9b None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:38.171472 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 305/599] 10.222.0.58 () {56 vars in 1148 bytes} [Tue Jul 26 13:33:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:38.172683 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddcbf10-4992-42ab-a21e-f91d8f15dd01 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:38.177319 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:38.184158 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:38.287169 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddcbf10-4992-42ab-a21e-f91d8f15dd01 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:38.290137 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c67973-85e3-46cd-800c-fb0390d26c9b None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:38.326750 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c7a694-a835-438f-959e-84ffe81a1a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:38.327839 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c7a694-a835-438f-959e-84ffe81a1a95 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:38.328529 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c7a694-a835-438f-959e-84ffe81a1a95 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:38.329016 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c7a694-a835-438f-959e-84ffe81a1a95 None admin] PATH_INFO: `/v3/domains` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:38.329955 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c7a694-a835-438f-959e-84ffe81a1a95 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:38.330548 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c7a694-a835-438f-959e-84ffe81a1a95 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:38.331184 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab408a86-0807-4d41-92e8-c6eb2653bbfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:38.332255 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab408a86-0807-4d41-92e8-c6eb2653bbfb None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:38.332608 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c7a694-a835-438f-959e-84ffe81a1a95 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:38.332713 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab408a86-0807-4d41-92e8-c6eb2653bbfb None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:38.333155 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab408a86-0807-4d41-92e8-c6eb2653bbfb None admin] PATH_INFO: `/v3/projects/service` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:38.342161 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab408a86-0807-4d41-92e8-c6eb2653bbfb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:38.344572 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab408a86-0807-4d41-92e8-c6eb2653bbfb None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:38.347695 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 306/600] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Jul 26 13:33:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:38.348760 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c7a694-a835-438f-959e-84ffe81a1a95 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:38.362189 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-ab408a86-0807-4d41-92e8-c6eb2653bbfb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 13:33:38.362189 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:38.362189 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:38.362189 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:38.362189 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:38.362189 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:38.362189 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:38.362189 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:38.362189 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:38.362189 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:38.362189 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:38.362189 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:38.362189 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 13:33:38.362189 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 13:33:38.362189 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 13:33:38.363998 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 13:33:38.363998 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:38.363998 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:38.363998 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:38.363998 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:38.363998 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:38.363998 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:38.363998 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:38.363998 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:38.363998 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:38.363998 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:38.363998 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:38.363998 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:38.363998 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:38.363998 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:38.363998 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:38.363998 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:38.365788 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 13:33:38.365788 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 13:33:38.365788 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 13:33:38.365788 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 13:33:38.365788 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 13:33:38.365788 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 13:33:38.365788 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 13:33:38.365788 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:33:38.368351 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 295/601] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Jul 26 13:33:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:38.369278 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab408a86-0807-4d41-92e8-c6eb2653bbfb None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:38.437014 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:38.447296 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:38.489866 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab408a86-0807-4d41-92e8-c6eb2653bbfb None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:38.492181 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c7a694-a835-438f-959e-84ffe81a1a95 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:38.532390 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90da946-68fc-49a5-b1b3-454db696d0bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:38.533080 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90da946-68fc-49a5-b1b3-454db696d0bf None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:38.533220 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81af6319-78e8-430f-bba6-650a6af5eceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:38.533411 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90da946-68fc-49a5-b1b3-454db696d0bf None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:38.533656 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90da946-68fc-49a5-b1b3-454db696d0bf None admin] PATH_INFO: `/v3/projects/service` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:38.534719 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81af6319-78e8-430f-bba6-650a6af5eceb None admin] REQUEST_METHOD: `PUT` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:38.535222 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81af6319-78e8-430f-bba6-650a6af5eceb None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:38.535693 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81af6319-78e8-430f-bba6-650a6af5eceb None admin] PATH_INFO: `/v3/projects/1e8bb4385c204210a1bc77b1f0d654f3/users/eaf065aef9a046b2adaec34816a336b7/roles/17e0f3c73a8a4a54b3e7b78878dfbf09` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:38.542695 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90da946-68fc-49a5-b1b3-454db696d0bf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:38.544067 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90da946-68fc-49a5-b1b3-454db696d0bf None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:38.554052 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81af6319-78e8-430f-bba6-650a6af5eceb None admin] RBAC: Authorizing `identity:create_grant(project_id=1e8bb4385c204210a1bc77b1f0d654f3, user_id=eaf065aef9a046b2adaec34816a336b7, role_id=17e0f3c73a8a4a54b3e7b78878dfbf09)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:38.555872 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81af6319-78e8-430f-bba6-650a6af5eceb None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:38.561244 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-c90da946-68fc-49a5-b1b3-454db696d0bf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 13:33:38.561244 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:38.561244 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:38.561244 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:38.561244 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:38.561244 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:38.561244 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:38.561244 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:38.561244 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:38.561244 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:38.561244 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:38.561244 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:38.561244 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 13:33:38.561244 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 13:33:38.561244 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 13:33:38.562219 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 13:33:38.562219 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:38.562219 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:38.562219 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:38.562219 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:38.562219 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:38.562219 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:38.562219 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:38.562219 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:38.562219 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:38.562219 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:38.562219 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:38.562219 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:38.562219 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:38.562219 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:38.562219 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:38.562219 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:38.563116 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 13:33:38.563116 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 13:33:38.563116 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 13:33:38.563116 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 13:33:38.563116 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 13:33:38.563116 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 13:33:38.563116 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 13:33:38.563116 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:33:38.563580 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 296/602] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Jul 26 13:33:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:38.564564 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90da946-68fc-49a5-b1b3-454db696d0bf None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:38.575625 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 307/603] 10.222.0.58 () {58 vars in 1439 bytes} [Tue Jul 26 13:33:38 2022] PUT /identity/v3/projects/1e8bb4385c204210a1bc77b1f0d654f3/users/eaf065aef9a046b2adaec34816a336b7/roles/17e0f3c73a8a4a54b3e7b78878dfbf09 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:33:38.576936 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81af6319-78e8-430f-bba6-650a6af5eceb None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:38.722811 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81af6319-78e8-430f-bba6-650a6af5eceb None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:38.748324 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:38.777651 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90da946-68fc-49a5-b1b3-454db696d0bf None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:38.789353 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb61d4fb-bce1-4414-b3f1-db87300dfe57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:38.790102 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb61d4fb-bce1-4414-b3f1-db87300dfe57 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:38.790400 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb61d4fb-bce1-4414-b3f1-db87300dfe57 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:38.790677 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb61d4fb-bce1-4414-b3f1-db87300dfe57 None admin] PATH_INFO: `/v3/projects` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:38.791558 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb61d4fb-bce1-4414-b3f1-db87300dfe57 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:38.791863 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb61d4fb-bce1-4414-b3f1-db87300dfe57 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:38.793627 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb61d4fb-bce1-4414-b3f1-db87300dfe57 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:38.808800 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 308/604] 10.222.0.58 () {56 vars in 1148 bytes} [Tue Jul 26 13:33:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:38.810514 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c762a97-908a-4d70-b97b-ef84be3e6d22 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:38.811016 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c762a97-908a-4d70-b97b-ef84be3e6d22 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:38.811354 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c762a97-908a-4d70-b97b-ef84be3e6d22 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:38.811494 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:38.812947 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 309/605] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:33:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:33:38.819496 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c762a97-908a-4d70-b97b-ef84be3e6d22 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:38.827674 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183ffb7d-c575-40b1-898e-77eb8ed787b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:38.828301 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183ffb7d-c575-40b1-898e-77eb8ed787b0 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:38.828578 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183ffb7d-c575-40b1-898e-77eb8ed787b0 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:38.829278 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183ffb7d-c575-40b1-898e-77eb8ed787b0 None admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:38.830293 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183ffb7d-c575-40b1-898e-77eb8ed787b0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:38.830400 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183ffb7d-c575-40b1-898e-77eb8ed787b0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:38.832158 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183ffb7d-c575-40b1-898e-77eb8ed787b0 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:38.846975 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 297/606] 10.222.0.58 () {56 vars in 1148 bytes} [Tue Jul 26 13:33:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:38.848679 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f2aa1d-3255-41b3-bc62-dbfe46036f6d None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:38.848970 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f2aa1d-3255-41b3-bc62-dbfe46036f6d None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:38.849249 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f2aa1d-3255-41b3-bc62-dbfe46036f6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:38.868707 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-d5f2aa1d-3255-41b3-bc62-dbfe46036f6d None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:33:38.870283 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-d5f2aa1d-3255-41b3-bc62-dbfe46036f6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:33:38.944763 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c762a97-908a-4d70-b97b-ef84be3e6d22 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:38.950363 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 298/607] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:33:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 13:33:38.952009 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f2aa1d-3255-41b3-bc62-dbfe46036f6d None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:39.005496 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0e7dc5-1ee1-4a1b-83ca-bc29e4601948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:39.006211 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0e7dc5-1ee1-4a1b-83ca-bc29e4601948 None admin] REQUEST_METHOD: `PUT` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:39.006488 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0e7dc5-1ee1-4a1b-83ca-bc29e4601948 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:39.006761 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0e7dc5-1ee1-4a1b-83ca-bc29e4601948 None admin] PATH_INFO: `/v3/projects/1e8bb4385c204210a1bc77b1f0d654f3/users/3223a78866d94c2ba66402911c2dfae9/roles/17e0f3c73a8a4a54b3e7b78878dfbf09` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:39.024082 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0e7dc5-1ee1-4a1b-83ca-bc29e4601948 None admin] RBAC: Authorizing `identity:create_grant(project_id=1e8bb4385c204210a1bc77b1f0d654f3, user_id=3223a78866d94c2ba66402911c2dfae9, role_id=17e0f3c73a8a4a54b3e7b78878dfbf09)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:39.025624 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0e7dc5-1ee1-4a1b-83ca-bc29e4601948 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:39.048349 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 310/608] 10.222.0.58 () {58 vars in 1439 bytes} [Tue Jul 26 13:33:38 2022] PUT /identity/v3/projects/1e8bb4385c204210a1bc77b1f0d654f3/users/3223a78866d94c2ba66402911c2dfae9/roles/17e0f3c73a8a4a54b3e7b78878dfbf09 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:33:39.049289 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0e7dc5-1ee1-4a1b-83ca-bc29e4601948 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:39.078952 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:39.115043 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f2aa1d-3255-41b3-bc62-dbfe46036f6d None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:39.148786 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:39.201954 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0e7dc5-1ee1-4a1b-83ca-bc29e4601948 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:39.205973 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc60ca7-c6e3-4b31-abee-69497b687061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:39.206672 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc60ca7-c6e3-4b31-abee-69497b687061 None admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:39.206953 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc60ca7-c6e3-4b31-abee-69497b687061 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:39.207227 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc60ca7-c6e3-4b31-abee-69497b687061 None admin] PATH_INFO: `/v3/projects/1e8bb4385c204210a1bc77b1f0d654f3/users/72e400f267fd488d8439426a81ba0a25/roles/17e0f3c73a8a4a54b3e7b78878dfbf09` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:39.251228 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc60ca7-c6e3-4b31-abee-69497b687061 None admin] RBAC: Authorizing `identity:create_grant(project_id=1e8bb4385c204210a1bc77b1f0d654f3, user_id=72e400f267fd488d8439426a81ba0a25, role_id=17e0f3c73a8a4a54b3e7b78878dfbf09)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:39.252783 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc60ca7-c6e3-4b31-abee-69497b687061 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:39.282764 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 299/609] 10.222.0.58 () {58 vars in 1439 bytes} [Tue Jul 26 13:33:38 2022] PUT /identity/v3/projects/1e8bb4385c204210a1bc77b1f0d654f3/users/72e400f267fd488d8439426a81ba0a25/roles/17e0f3c73a8a4a54b3e7b78878dfbf09 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:33:39.285829 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbcf8e1-e546-4025-a128-7b8809ca8bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:39.286856 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbcf8e1-e546-4025-a128-7b8809ca8bed None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:39.287362 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbcf8e1-e546-4025-a128-7b8809ca8bed None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:39.288514 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbcf8e1-e546-4025-a128-7b8809ca8bed None admin] PATH_INFO: `/v3/roles/service` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:39.304931 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbcf8e1-e546-4025-a128-7b8809ca8bed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:33:39.311750 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-abbcf8e1-e546-4025-a128-7b8809ca8bed None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 13:33:39.311750 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:39.311750 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:39.311750 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:39.311750 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:39.311750 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:39.311750 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:39.311750 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:39.311750 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:39.311750 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:39.311750 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:39.311750 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:39.311750 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:33:39.311750 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:33:39.311750 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:33:39.311750 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:33:39.312782 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:33:39.312782 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:33:39.312782 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:33:39.312782 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:33:39.312782 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:39.312782 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:39.312782 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:39.312782 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:39.312782 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:39.312782 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:39.312782 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:39.312782 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:39.312782 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:39.312782 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:39.312782 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:39.312782 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:39.313651 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:39.313651 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:39.313651 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:39.313651 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:39.313651 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:33:39.313651 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:33:39.313651 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:33:39.313651 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:33:39.313651 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:33:39.313651 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:33:39.313651 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 13:33:39.313651 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:33:39.316760 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 311/610] 10.222.0.58 () {56 vars in 1102 bytes} [Tue Jul 26 13:33:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:39.322381 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc60ca7-c6e3-4b31-abee-69497b687061 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:39.456051 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc60ca7-c6e3-4b31-abee-69497b687061 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:39.496364 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1268d86d-559d-4651-a8c9-3d8800924aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:39.497434 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1268d86d-559d-4651-a8c9-3d8800924aa5 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:39.498040 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1268d86d-559d-4651-a8c9-3d8800924aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:39.498541 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1268d86d-559d-4651-a8c9-3d8800924aa5 None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:39.499494 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1268d86d-559d-4651-a8c9-3d8800924aa5 None admin] RBAC: Adding query filter params (name=service) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:39.500006 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1268d86d-559d-4651-a8c9-3d8800924aa5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:39.501630 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1268d86d-559d-4651-a8c9-3d8800924aa5 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:39.512034 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 300/611] 10.222.0.58 () {56 vars in 1103 bytes} [Tue Jul 26 13:33:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:33:39.521287 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbcf8e1-e546-4025-a128-7b8809ca8bed None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:39.627075 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7512f5f0-4e19-4d0a-a257-af068430efe2 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:39.627680 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7512f5f0-4e19-4d0a-a257-af068430efe2 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:39.627957 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7512f5f0-4e19-4d0a-a257-af068430efe2 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:39.629509 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 301/612] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:33:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 13:33:39.635745 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1163ca8-eb8c-44be-bfed-524c6a1d7d4c None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:39.636049 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1163ca8-eb8c-44be-bfed-524c6a1d7d4c None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:39.636302 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1163ca8-eb8c-44be-bfed-524c6a1d7d4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:39.641241 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:39.657825 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbcf8e1-e546-4025-a128-7b8809ca8bed None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:39.671461 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-a1163ca8-eb8c-44be-bfed-524c6a1d7d4c None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:33:39.672688 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-a1163ca8-eb8c-44be-bfed-524c6a1d7d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:33:39.712336 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72993885-9021-42e9-a5ab-ee681bda3650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:39.713341 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72993885-9021-42e9-a5ab-ee681bda3650 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:39.713970 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72993885-9021-42e9-a5ab-ee681bda3650 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:39.714677 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72993885-9021-42e9-a5ab-ee681bda3650 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:39.717763 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:39.731722 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72993885-9021-42e9-a5ab-ee681bda3650 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:39.733603 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72993885-9021-42e9-a5ab-ee681bda3650 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:39.764681 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-72993885-9021-42e9-a5ab-ee681bda3650 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:39.764681 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:39.764681 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:39.764681 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:39.764681 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:39.764681 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:39.764681 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:39.764681 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:39.764681 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:39.764681 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:39.764681 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:39.764681 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:39.764681 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 13:33:39.764681 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 13:33:39.764681 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 13:33:39.765698 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 13:33:39.765698 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:39.765698 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:39.765698 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:39.765698 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:39.765698 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:39.765698 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:39.765698 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:39.765698 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:39.765698 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:39.765698 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:39.765698 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:39.765698 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:39.765698 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:39.765698 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:39.765698 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:39.765698 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:39.767155 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 13:33:39.767155 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 13:33:39.767155 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:39.767155 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:33:39.767155 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 312/613] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:33:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:39.772098 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72993885-9021-42e9-a5ab-ee681bda3650 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:39.803192 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 302/614] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:33:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 13:33:39.842002 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1163ca8-eb8c-44be-bfed-524c6a1d7d4c None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:39.942239 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72993885-9021-42e9-a5ab-ee681bda3650 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:40.017554 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6486c514-6331-4961-9f09-9af6f3cef0ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:40.018770 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6486c514-6331-4961-9f09-9af6f3cef0ba None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:40.019761 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6486c514-6331-4961-9f09-9af6f3cef0ba None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:40.020575 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6486c514-6331-4961-9f09-9af6f3cef0ba None admin] PATH_INFO: `/v3/domains` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:40.021532 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6486c514-6331-4961-9f09-9af6f3cef0ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:40.022344 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6486c514-6331-4961-9f09-9af6f3cef0ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:40.026761 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6486c514-6331-4961-9f09-9af6f3cef0ba None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:40.030559 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:40.052392 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 313/615] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Jul 26 13:33:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:33:40.056652 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1163ca8-eb8c-44be-bfed-524c6a1d7d4c None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:40.062637 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6486c514-6331-4961-9f09-9af6f3cef0ba None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:40.094420 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6a18ca-94ad-4d35-a543-4d7fa74419ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:40.094420 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a18ca-94ad-4d35-a543-4d7fa74419ce None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:40.094701 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a18ca-94ad-4d35-a543-4d7fa74419ce None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:40.094952 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a18ca-94ad-4d35-a543-4d7fa74419ce None admin] PATH_INFO: `/v3/roles/service` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:40.106727 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6a18ca-94ad-4d35-a543-4d7fa74419ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:33:40.111661 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-dd6a18ca-94ad-4d35-a543-4d7fa74419ce None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 13:33:40.111661 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:40.111661 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:40.111661 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:40.111661 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:40.111661 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:40.111661 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:40.111661 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:40.111661 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:40.111661 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:40.111661 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:40.111661 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:40.111661 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:33:40.111661 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:33:40.111661 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:33:40.111661 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:33:40.112698 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:33:40.112698 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:33:40.112698 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:33:40.112698 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:33:40.112698 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:40.112698 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:40.112698 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:40.112698 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:40.112698 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:40.112698 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:40.112698 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:40.112698 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:40.112698 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:40.112698 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:40.112698 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:40.112698 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:40.114087 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:40.114087 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:40.114087 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:40.114087 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:40.114087 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:33:40.114087 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:33:40.114087 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:33:40.114087 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:33:40.114087 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:33:40.114087 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:33:40.114087 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 13:33:40.114087 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:33:40.114087 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 303/616] 10.222.0.58 () {56 vars in 1102 bytes} [Tue Jul 26 13:33:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:40.119571 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6a18ca-94ad-4d35-a543-4d7fa74419ce None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:40.142375 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:40.231493 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6a18ca-94ad-4d35-a543-4d7fa74419ce None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:40.235505 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6486c514-6331-4961-9f09-9af6f3cef0ba None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:40.267998 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5757476-081f-48ec-b8fb-8cb58dac2fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:40.268661 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5757476-081f-48ec-b8fb-8cb58dac2fce None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:40.268942 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5757476-081f-48ec-b8fb-8cb58dac2fce None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:40.269217 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5757476-081f-48ec-b8fb-8cb58dac2fce None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:40.270211 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5757476-081f-48ec-b8fb-8cb58dac2fce None admin] RBAC: Adding query filter params (name=service) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:40.270345 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5757476-081f-48ec-b8fb-8cb58dac2fce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:40.271653 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5757476-081f-48ec-b8fb-8cb58dac2fce None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:40.281073 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 304/617] 10.222.0.58 () {56 vars in 1103 bytes} [Tue Jul 26 13:33:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:33:40.289671 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5757476-081f-48ec-b8fb-8cb58dac2fce None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:40.319318 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e4057c-b914-4a06-b196-e23f603610ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:40.320026 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e4057c-b914-4a06-b196-e23f603610ce None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:40.320318 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e4057c-b914-4a06-b196-e23f603610ce None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:40.320595 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e4057c-b914-4a06-b196-e23f603610ce None admin] PATH_INFO: `/v3/users/swift` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:40.337424 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e4057c-b914-4a06-b196-e23f603610ce None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:40.338990 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e4057c-b914-4a06-b196-e23f603610ce None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:40.357912 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-c3e4057c-b914-4a06-b196-e23f603610ce None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 26 13:33:40.357912 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:40.357912 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:40.357912 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:40.357912 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:40.357912 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:40.357912 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:40.357912 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:40.357912 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:40.357912 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:40.357912 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:40.357912 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:40.357912 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 13:33:40.357912 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 13:33:40.357912 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 13:33:40.357912 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 13:33:40.358994 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:40.358994 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:40.358994 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 13:33:40.358994 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 13:33:40.358994 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 13:33:40.358994 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 13:33:40.358994 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:40.358994 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:40.358994 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:40.358994 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:40.358994 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:40.358994 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:40.358994 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:40.358994 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:40.358994 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:40.358994 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:40.358994 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:40.358994 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:40.360324 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:40.360324 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:40.360324 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 13:33:40.360324 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 13:33:40.360324 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 13:33:40.360324 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 13:33:40.360324 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 13:33:40.360324 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 13:33:40.360324 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 26 13:33:40.360324 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:33:40.360324 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 314/618] 10.222.0.58 () {56 vars in 1096 bytes} [Tue Jul 26 13:33:40 2022] GET /identity/v3/users/swift => generated 83 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:40.366046 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e4057c-b914-4a06-b196-e23f603610ce None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:40.472916 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:40.489941 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:40.498679 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5757476-081f-48ec-b8fb-8cb58dac2fce None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:40.536291 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db399d68-f3c5-4bc4-b8ee-919a911b9a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:40.536942 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db399d68-f3c5-4bc4-b8ee-919a911b9a81 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:40.537225 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db399d68-f3c5-4bc4-b8ee-919a911b9a81 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:40.537535 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db399d68-f3c5-4bc4-b8ee-919a911b9a81 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:40.551140 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db399d68-f3c5-4bc4-b8ee-919a911b9a81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:40.552614 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db399d68-f3c5-4bc4-b8ee-919a911b9a81 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:40.568350 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-db399d68-f3c5-4bc4-b8ee-919a911b9a81 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:40.568350 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:40.568350 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:40.568350 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:40.568350 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:40.568350 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:40.568350 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:40.568350 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:40.568350 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:40.568350 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:40.568350 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:40.568350 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:40.568350 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 13:33:40.568350 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 13:33:40.568350 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 13:33:40.569314 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 13:33:40.569314 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:40.569314 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:40.569314 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:40.569314 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:40.569314 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:40.569314 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:40.569314 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:40.569314 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:40.569314 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:40.569314 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:40.569314 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:40.569314 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:40.569314 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:40.569314 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:40.569314 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:40.569314 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:40.571000 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 13:33:40.571000 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 13:33:40.571000 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:40.571000 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:33:40.571000 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 305/619] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:33:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:40.575383 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db399d68-f3c5-4bc4-b8ee-919a911b9a81 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:40.617371 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e4057c-b914-4a06-b196-e23f603610ce None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:40.687119 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db399d68-f3c5-4bc4-b8ee-919a911b9a81 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:40.719097 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d598c7-de9b-41e9-b6e4-ee1c8da21095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:40.719742 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d598c7-de9b-41e9-b6e4-ee1c8da21095 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:40.720036 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d598c7-de9b-41e9-b6e4-ee1c8da21095 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:40.720291 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d598c7-de9b-41e9-b6e4-ee1c8da21095 None admin] PATH_INFO: `/v3/users` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:40.721169 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d598c7-de9b-41e9-b6e4-ee1c8da21095 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:40.721501 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d598c7-de9b-41e9-b6e4-ee1c8da21095 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:40.722936 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d598c7-de9b-41e9-b6e4-ee1c8da21095 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:40.724914 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c2a0e9-4204-4158-b8a8-71c41d97e0b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:40.725584 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c2a0e9-4204-4158-b8a8-71c41d97e0b6 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:40.725873 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c2a0e9-4204-4158-b8a8-71c41d97e0b6 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:40.726147 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c2a0e9-4204-4158-b8a8-71c41d97e0b6 None admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:40.726873 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c2a0e9-4204-4158-b8a8-71c41d97e0b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:40.727183 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c2a0e9-4204-4158-b8a8-71c41d97e0b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:40.729173 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c2a0e9-4204-4158-b8a8-71c41d97e0b6 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:40.731097 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:40.744606 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 306/620] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Jul 26 13:33:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:33:40.750170 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c2a0e9-4204-4158-b8a8-71c41d97e0b6 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:40.756937 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 315/621] 10.222.0.58 () {56 vars in 1135 bytes} [Tue Jul 26 13:33:40 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 391 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:40.762077 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d598c7-de9b-41e9-b6e4-ee1c8da21095 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:40.878412 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c2a0e9-4204-4158-b8a8-71c41d97e0b6 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:40.904561 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d598c7-de9b-41e9-b6e4-ee1c8da21095 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:40.927086 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:40.959584 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877ae05a-29f2-46b3-b541-4e38fc50a369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:40.960233 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877ae05a-29f2-46b3-b541-4e38fc50a369 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:40.960514 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877ae05a-29f2-46b3-b541-4e38fc50a369 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:40.960785 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877ae05a-29f2-46b3-b541-4e38fc50a369 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:40.972222 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91da203f-0cdf-4c3f-8f6f-ab3a1575bd56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:40.972883 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91da203f-0cdf-4c3f-8f6f-ab3a1575bd56 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:40.973161 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91da203f-0cdf-4c3f-8f6f-ab3a1575bd56 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:40.973430 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91da203f-0cdf-4c3f-8f6f-ab3a1575bd56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:40.975155 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877ae05a-29f2-46b3-b541-4e38fc50a369 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:40.976591 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877ae05a-29f2-46b3-b541-4e38fc50a369 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:40.993749 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91da203f-0cdf-4c3f-8f6f-ab3a1575bd56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:40.995648 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91da203f-0cdf-4c3f-8f6f-ab3a1575bd56 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:40.995878 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-877ae05a-29f2-46b3-b541-4e38fc50a369 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 26 13:33:40.995878 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:40.995878 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:40.995878 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:40.995878 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:40.995878 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:40.995878 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:40.995878 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:40.995878 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:40.995878 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:40.995878 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:40.995878 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:40.995878 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 13:33:40.995878 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 13:33:40.995878 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 13:33:40.995878 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 13:33:40.996797 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:40.996797 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:40.996797 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 13:33:40.996797 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 13:33:40.996797 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 13:33:40.996797 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 13:33:40.996797 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:40.996797 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:40.996797 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:40.996797 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:40.996797 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:40.996797 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:40.996797 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:40.996797 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:40.996797 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:40.996797 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:40.996797 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:40.996797 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:41.002067 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:41.002067 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:41.002067 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 13:33:41.002067 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 13:33:41.002067 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 13:33:41.002067 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 13:33:41.002067 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 13:33:41.002067 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 13:33:41.002067 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 26 13:33:41.002067 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:33:41.002067 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 307/622] 10.222.0.58 () {56 vars in 1093 bytes} [Tue Jul 26 13:33:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 13:33:41.002678 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877ae05a-29f2-46b3-b541-4e38fc50a369 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:41.028202 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-91da203f-0cdf-4c3f-8f6f-ab3a1575bd56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:41.028202 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:41.028202 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:41.028202 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:41.028202 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:41.028202 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:41.028202 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:41.028202 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:41.028202 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:41.028202 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:41.028202 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:41.028202 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:41.028202 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 13:33:41.028202 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 13:33:41.028202 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 13:33:41.029173 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 13:33:41.029173 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:41.029173 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:41.029173 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:41.029173 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:41.029173 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:41.029173 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:41.029173 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:41.029173 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:41.029173 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:41.029173 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:41.029173 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:41.029173 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:41.029173 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:41.029173 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:41.029173 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:41.029173 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:41.030198 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 13:33:41.030198 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 13:33:41.030198 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:41.030198 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:33:41.036874 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 316/623] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:33:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:41.039934 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91da203f-0cdf-4c3f-8f6f-ab3a1575bd56 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:41.122210 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:41.183139 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877ae05a-29f2-46b3-b541-4e38fc50a369 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:41.185114 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91da203f-0cdf-4c3f-8f6f-ab3a1575bd56 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:41.220535 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23fd1ea-0094-4f27-b265-2fa0525d547c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:41.221232 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23fd1ea-0094-4f27-b265-2fa0525d547c None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:41.221547 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23fd1ea-0094-4f27-b265-2fa0525d547c None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:41.221828 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23fd1ea-0094-4f27-b265-2fa0525d547c None admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:41.222634 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23fd1ea-0094-4f27-b265-2fa0525d547c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:41.222945 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23fd1ea-0094-4f27-b265-2fa0525d547c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:41.223464 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae7bf86-3824-4e7b-9479-b898c61e02f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:41.224330 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae7bf86-3824-4e7b-9479-b898c61e02f9 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:41.224449 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23fd1ea-0094-4f27-b265-2fa0525d547c None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:41.224927 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae7bf86-3824-4e7b-9479-b898c61e02f9 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:41.225411 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae7bf86-3824-4e7b-9479-b898c61e02f9 None admin] PATH_INFO: `/v3/domains` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:41.226581 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae7bf86-3824-4e7b-9479-b898c61e02f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:41.227120 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae7bf86-3824-4e7b-9479-b898c61e02f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:41.228816 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae7bf86-3824-4e7b-9479-b898c61e02f9 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:41.242253 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 317/624] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Jul 26 13:33:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:41.244583 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3360cbc4-7c7c-4f44-b8b3-413b1d4bfe25 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:41.245101 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3360cbc4-7c7c-4f44-b8b3-413b1d4bfe25 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:41.245825 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3360cbc4-7c7c-4f44-b8b3-413b1d4bfe25 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:41.246496 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 308/625] 10.222.0.58 () {56 vars in 1133 bytes} [Tue Jul 26 13:33:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:41.247882 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23fd1ea-0094-4f27-b265-2fa0525d547c None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:41.248787 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 318/626] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:33:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:33:41.257593 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3360cbc4-7c7c-4f44-b8b3-413b1d4bfe25 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:41.287268 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:41.360307 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23fd1ea-0094-4f27-b265-2fa0525d547c None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:41.375615 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:41.377139 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3360cbc4-7c7c-4f44-b8b3-413b1d4bfe25 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:41.407142 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033e43ce-ca50-4d41-ad6d-735ee2907882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:41.408468 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e43ce-ca50-4d41-ad6d-735ee2907882 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:41.408857 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e43ce-ca50-4d41-ad6d-735ee2907882 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:41.409231 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e43ce-ca50-4d41-ad6d-735ee2907882 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:41.420978 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033e43ce-ca50-4d41-ad6d-735ee2907882 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:41.422920 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033e43ce-ca50-4d41-ad6d-735ee2907882 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:41.428465 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b11b483-6f74-4a19-a281-1027c26698b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:41.429188 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b11b483-6f74-4a19-a281-1027c26698b2 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:41.429568 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b11b483-6f74-4a19-a281-1027c26698b2 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:41.429916 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b11b483-6f74-4a19-a281-1027c26698b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:41.438344 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-033e43ce-ca50-4d41-ad6d-735ee2907882 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 13:33:41.438344 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:41.438344 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:41.438344 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:41.438344 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:41.438344 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:41.438344 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:41.438344 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:41.438344 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:41.438344 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:41.438344 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:41.438344 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:41.438344 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 13:33:41.438344 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 13:33:41.438344 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 13:33:41.439306 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 13:33:41.439306 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:41.439306 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:41.439306 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:41.439306 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:41.439306 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:41.439306 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:41.439306 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:41.439306 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:41.439306 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:41.439306 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:41.439306 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:41.439306 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:41.439306 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:41.439306 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:41.439306 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:41.439306 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:41.440233 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 13:33:41.440233 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 13:33:41.440233 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 13:33:41.440233 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 13:33:41.440233 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 13:33:41.440233 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 13:33:41.440233 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 13:33:41.440233 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:33:41.442687 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 309/627] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Jul 26 13:33:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:41.444317 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e88316f-c673-4c67-89f6-3dcb0cf9840d None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:41.444768 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e88316f-c673-4c67-89f6-3dcb0cf9840d None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:41.445255 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e88316f-c673-4c67-89f6-3dcb0cf9840d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:41.466359 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-0e88316f-c673-4c67-89f6-3dcb0cf9840d None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:33:41.468026 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-0e88316f-c673-4c67-89f6-3dcb0cf9840d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:33:41.509191 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b11b483-6f74-4a19-a281-1027c26698b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:41.511360 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b11b483-6f74-4a19-a281-1027c26698b2 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:41.539119 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:41.550948 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 310/628] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:33:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 13:33:41.552226 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e88316f-c673-4c67-89f6-3dcb0cf9840d None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:41.618766 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 319/629] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Jul 26 13:33:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 13:33:41.623519 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b11b483-6f74-4a19-a281-1027c26698b2 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:41.672013 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e88316f-c673-4c67-89f6-3dcb0cf9840d None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:41.674506 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:41.714505 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb92fd2c-ca72-44d6-925e-8d3d2f99952c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:41.715571 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb92fd2c-ca72-44d6-925e-8d3d2f99952c None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:41.716118 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb92fd2c-ca72-44d6-925e-8d3d2f99952c None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:41.716612 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb92fd2c-ca72-44d6-925e-8d3d2f99952c None admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:41.717770 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb92fd2c-ca72-44d6-925e-8d3d2f99952c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:41.718389 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb92fd2c-ca72-44d6-925e-8d3d2f99952c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:41.720188 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb92fd2c-ca72-44d6-925e-8d3d2f99952c None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:41.727580 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b11b483-6f74-4a19-a281-1027c26698b2 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:41.734992 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 311/630] 10.222.0.58 () {56 vars in 1148 bytes} [Tue Jul 26 13:33:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:41.736208 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb92fd2c-ca72-44d6-925e-8d3d2f99952c None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:41.756999 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:41.767106 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3016ce-78af-4222-ab23-b80d1dbcb158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:41.767713 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3016ce-78af-4222-ab23-b80d1dbcb158 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:41.768004 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3016ce-78af-4222-ab23-b80d1dbcb158 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:41.768285 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3016ce-78af-4222-ab23-b80d1dbcb158 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:41.780204 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3016ce-78af-4222-ab23-b80d1dbcb158 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:33:41.785462 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-bb3016ce-78af-4222-ab23-b80d1dbcb158 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 13:33:41.785462 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:41.785462 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:41.785462 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:41.785462 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:41.785462 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:41.785462 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:41.785462 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:41.785462 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:41.785462 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:41.785462 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:41.785462 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:41.785462 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:33:41.785462 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:33:41.785462 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:33:41.785462 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:33:41.786484 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:33:41.786484 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:33:41.786484 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:33:41.786484 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:33:41.786484 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:41.786484 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:41.786484 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:41.786484 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:41.786484 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:41.786484 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:41.786484 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:41.786484 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:41.786484 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:41.786484 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:41.786484 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:41.786484 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:41.787339 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:41.787339 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:41.787339 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:41.787339 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:41.787339 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:33:41.787339 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:33:41.787339 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:33:41.787339 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:33:41.787339 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:33:41.787339 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:33:41.787339 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 13:33:41.787339 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:33:41.787339 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 320/631] 10.222.0.58 () {56 vars in 1102 bytes} [Tue Jul 26 13:33:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:41.788053 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3016ce-78af-4222-ab23-b80d1dbcb158 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:41.862808 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb92fd2c-ca72-44d6-925e-8d3d2f99952c None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:41.905879 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ced33b7-e8a4-4177-adfc-51b4cbb7a391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:41.906529 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ced33b7-e8a4-4177-adfc-51b4cbb7a391 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:41.906813 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ced33b7-e8a4-4177-adfc-51b4cbb7a391 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:41.907094 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ced33b7-e8a4-4177-adfc-51b4cbb7a391 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:41.909186 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3016ce-78af-4222-ab23-b80d1dbcb158 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:41.918476 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ced33b7-e8a4-4177-adfc-51b4cbb7a391 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:41.919917 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ced33b7-e8a4-4177-adfc-51b4cbb7a391 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:41.937223 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-3ced33b7-e8a4-4177-adfc-51b4cbb7a391 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:41.937223 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:41.937223 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:41.937223 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:41.937223 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:41.937223 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:41.937223 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:41.937223 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:41.937223 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:41.937223 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:41.937223 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:41.937223 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:41.937223 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 13:33:41.937223 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 13:33:41.937223 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 13:33:41.938555 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 13:33:41.938555 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:41.938555 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:41.938555 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:41.938555 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:41.938555 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:41.938555 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:41.938555 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:41.938555 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:41.938555 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:41.938555 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:41.938555 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:41.938555 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:41.938555 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:41.938555 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:41.938555 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:41.938555 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:41.940779 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 13:33:41.940779 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 13:33:41.940779 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:41.940779 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:33:41.940779 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 312/632] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:33:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:41.940779 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ced33b7-e8a4-4177-adfc-51b4cbb7a391 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:41.951503 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc7739d-df1c-466e-b45e-ba247a230cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:41.952292 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc7739d-df1c-466e-b45e-ba247a230cac None admin] REQUEST_METHOD: `PUT` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:41.952577 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc7739d-df1c-466e-b45e-ba247a230cac None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:41.952939 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc7739d-df1c-466e-b45e-ba247a230cac None admin] PATH_INFO: `/v3/projects/1e8bb4385c204210a1bc77b1f0d654f3/users/3ab2819bac844163ae4305e4f2186605/roles/17e0f3c73a8a4a54b3e7b78878dfbf09` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:41.958085 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:41.972381 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc7739d-df1c-466e-b45e-ba247a230cac None admin] RBAC: Authorizing `identity:create_grant(project_id=1e8bb4385c204210a1bc77b1f0d654f3, user_id=3ab2819bac844163ae4305e4f2186605, role_id=17e0f3c73a8a4a54b3e7b78878dfbf09)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:41.973920 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc7739d-df1c-466e-b45e-ba247a230cac None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:41.988147 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 321/633] 10.222.0.58 () {58 vars in 1439 bytes} [Tue Jul 26 13:33:41 2022] PUT /identity/v3/projects/1e8bb4385c204210a1bc77b1f0d654f3/users/3ab2819bac844163ae4305e4f2186605/roles/17e0f3c73a8a4a54b3e7b78878dfbf09 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:33:41.989410 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dc5634-2668-4f92-ae2f-35f3d6759e57 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:41.989725 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dc5634-2668-4f92-ae2f-35f3d6759e57 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:41.990001 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dc5634-2668-4f92-ae2f-35f3d6759e57 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:41.991277 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 322/634] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:33:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:33:41.993108 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd8642e-d16e-4d97-bc47-5ac0ebce5239 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:41.993108 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd8642e-d16e-4d97-bc47-5ac0ebce5239 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:41.993108 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd8642e-d16e-4d97-bc47-5ac0ebce5239 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:41.994977 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 323/635] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:33:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:33:41.998469 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd8642e-d16e-4d97-bc47-5ac0ebce5239 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:42.052749 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:42.100781 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ced33b7-e8a4-4177-adfc-51b4cbb7a391 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:42.155786 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea5afea-823d-4ec4-a719-1d4e2d225d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:42.156427 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea5afea-823d-4ec4-a719-1d4e2d225d46 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:42.156711 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea5afea-823d-4ec4-a719-1d4e2d225d46 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:42.156987 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea5afea-823d-4ec4-a719-1d4e2d225d46 None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:42.157800 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea5afea-823d-4ec4-a719-1d4e2d225d46 None admin] RBAC: Adding query filter params (name=service) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:42.158118 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea5afea-823d-4ec4-a719-1d4e2d225d46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:42.159527 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea5afea-823d-4ec4-a719-1d4e2d225d46 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:42.162252 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd8642e-d16e-4d97-bc47-5ac0ebce5239 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:42.169667 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 313/636] 10.222.0.58 () {56 vars in 1103 bytes} [Tue Jul 26 13:33:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:42.171091 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e949388-782a-47fb-a4a2-532498eff34e None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:42.171508 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e949388-782a-47fb-a4a2-532498eff34e None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:42.171693 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e949388-782a-47fb-a4a2-532498eff34e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:42.191856 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-6e949388-782a-47fb-a4a2-532498eff34e None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:33:42.193186 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-6e949388-782a-47fb-a4a2-532498eff34e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:33:42.203929 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1102ac1f-b506-466b-8864-769037cb6ca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:42.204924 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1102ac1f-b506-466b-8864-769037cb6ca5 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:42.205212 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1102ac1f-b506-466b-8864-769037cb6ca5 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:42.205508 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1102ac1f-b506-466b-8864-769037cb6ca5 None admin] PATH_INFO: `/v3/domains` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:42.206225 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1102ac1f-b506-466b-8864-769037cb6ca5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:42.206531 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1102ac1f-b506-466b-8864-769037cb6ca5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:42.207904 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1102ac1f-b506-466b-8864-769037cb6ca5 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:42.219588 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 324/637] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Jul 26 13:33:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:42.221000 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94942669-4205-4187-84fd-8e910eb96889 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:42.221281 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94942669-4205-4187-84fd-8e910eb96889 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:42.221982 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94942669-4205-4187-84fd-8e910eb96889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:42.242526 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-94942669-4205-4187-84fd-8e910eb96889 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:33:42.244335 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-94942669-4205-4187-84fd-8e910eb96889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:33:42.277880 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 314/638] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:33:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 13:33:42.278919 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e949388-782a-47fb-a4a2-532498eff34e None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:42.300293 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:42.326700 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 325/639] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:33:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 13:33:42.327814 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94942669-4205-4187-84fd-8e910eb96889 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:42.362118 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:42.426541 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e949388-782a-47fb-a4a2-532498eff34e None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:42.445188 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94942669-4205-4187-84fd-8e910eb96889 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:42.465677 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53a0579-8b68-4add-97f2-19bbd6bd8ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:42.466340 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53a0579-8b68-4add-97f2-19bbd6bd8ef4 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:42.466624 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53a0579-8b68-4add-97f2-19bbd6bd8ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:42.466896 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53a0579-8b68-4add-97f2-19bbd6bd8ef4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:42.481779 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53a0579-8b68-4add-97f2-19bbd6bd8ef4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:42.483245 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53a0579-8b68-4add-97f2-19bbd6bd8ef4 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:42.484545 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c24acc0-dd4a-4855-933a-b0ff4dba677c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:42.485205 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c24acc0-dd4a-4855-933a-b0ff4dba677c None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:42.485507 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c24acc0-dd4a-4855-933a-b0ff4dba677c None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:42.485787 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c24acc0-dd4a-4855-933a-b0ff4dba677c None admin] PATH_INFO: `/v3/projects/service` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:42.493039 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c24acc0-dd4a-4855-933a-b0ff4dba677c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:42.495131 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c24acc0-dd4a-4855-933a-b0ff4dba677c None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:42.500180 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-f53a0579-8b68-4add-97f2-19bbd6bd8ef4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:42.500180 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:42.500180 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:42.500180 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:42.500180 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:42.500180 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:42.500180 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:42.500180 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:42.500180 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:42.500180 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:42.500180 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:42.500180 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:42.500180 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 13:33:42.500180 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 13:33:42.500180 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 13:33:42.502574 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 13:33:42.502574 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:42.502574 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:42.502574 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:42.502574 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:42.502574 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:42.502574 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:42.502574 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:42.502574 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:42.502574 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:42.502574 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:42.502574 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:42.502574 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:42.502574 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:42.502574 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:42.502574 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:42.502574 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:42.503512 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 13:33:42.503512 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 13:33:42.503512 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:42.503512 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:33:42.503512 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 315/640] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:33:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:42.503512 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53a0579-8b68-4add-97f2-19bbd6bd8ef4 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:42.514538 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-0c24acc0-dd4a-4855-933a-b0ff4dba677c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 13:33:42.514538 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:42.514538 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:42.514538 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:42.514538 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:42.514538 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:42.514538 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:42.514538 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:42.514538 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:42.514538 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:42.514538 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:42.514538 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:42.514538 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 13:33:42.514538 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 13:33:42.514538 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 13:33:42.515614 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 13:33:42.515614 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:42.515614 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:42.515614 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:42.515614 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:42.515614 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:42.515614 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:42.515614 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:42.515614 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:42.515614 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:42.515614 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:42.515614 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:42.515614 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:42.515614 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:42.515614 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:42.515614 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:42.515614 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:42.516946 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 13:33:42.516946 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 13:33:42.516946 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 13:33:42.516946 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 13:33:42.516946 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 13:33:42.516946 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 13:33:42.516946 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 13:33:42.516946 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:33:42.516946 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 326/641] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Jul 26 13:33:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:42.517866 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c24acc0-dd4a-4855-933a-b0ff4dba677c None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:42.585303 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:42.616306 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:42.636549 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c24acc0-dd4a-4855-933a-b0ff4dba677c None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:42.640331 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53a0579-8b68-4add-97f2-19bbd6bd8ef4 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:42.673432 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908d8adb-c3c3-43a2-8908-73817a0a61b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:42.674095 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908d8adb-c3c3-43a2-8908-73817a0a61b7 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:42.674372 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908d8adb-c3c3-43a2-8908-73817a0a61b7 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:42.674654 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908d8adb-c3c3-43a2-8908-73817a0a61b7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:42.679192 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bfab0c-9a17-42dd-ba2d-ecefaa3a53ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:42.679842 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bfab0c-9a17-42dd-ba2d-ecefaa3a53ed None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:42.680124 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bfab0c-9a17-42dd-ba2d-ecefaa3a53ed None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:42.680394 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bfab0c-9a17-42dd-ba2d-ecefaa3a53ed None admin] PATH_INFO: `/v3/roles/service` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:42.686566 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908d8adb-c3c3-43a2-8908-73817a0a61b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:33:42.691594 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-908d8adb-c3c3-43a2-8908-73817a0a61b7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 13:33:42.691594 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:42.691594 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:42.691594 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:42.691594 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:42.691594 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:42.691594 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:42.691594 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:42.691594 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:42.691594 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:42.691594 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:42.691594 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:42.691594 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:33:42.691594 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:33:42.691594 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:33:42.691594 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:33:42.692826 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:33:42.692826 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:33:42.692826 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:33:42.692826 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:33:42.692826 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:42.692826 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:42.692826 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:42.692826 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:42.692826 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:42.692826 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:42.692826 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:42.692826 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:42.692826 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:42.692826 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:42.692826 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:42.692826 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:42.694005 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:42.694005 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:42.694005 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:42.694005 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:42.694005 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:33:42.694005 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:33:42.694005 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:33:42.694005 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:33:42.694005 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:33:42.694005 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:33:42.694005 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 13:33:42.694005 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:33:42.697735 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 327/642] 10.222.0.58 () {56 vars in 1102 bytes} [Tue Jul 26 13:33:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:42.698247 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bfab0c-9a17-42dd-ba2d-ecefaa3a53ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:33:42.698657 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908d8adb-c3c3-43a2-8908-73817a0a61b7 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:42.703706 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-33bfab0c-9a17-42dd-ba2d-ecefaa3a53ed None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 13:33:42.703706 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:42.703706 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:42.703706 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:42.703706 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:42.703706 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:42.703706 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:42.703706 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:42.703706 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:42.703706 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:42.703706 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:42.703706 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:42.703706 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:33:42.703706 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:33:42.703706 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:33:42.703706 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:33:42.704849 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:33:42.704849 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:33:42.704849 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:33:42.704849 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:33:42.704849 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:42.704849 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:42.704849 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:42.704849 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:42.704849 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:42.704849 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:42.704849 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:42.704849 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:42.704849 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:42.704849 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:42.704849 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:42.704849 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:42.710780 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:42.710780 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:42.710780 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:42.710780 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:42.710780 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:33:42.710780 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:33:42.710780 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:33:42.710780 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:33:42.710780 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:33:42.710780 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:33:42.710780 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 13:33:42.710780 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:33:42.710780 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 316/643] 10.222.0.58 () {56 vars in 1102 bytes} [Tue Jul 26 13:33:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:42.710780 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bfab0c-9a17-42dd-ba2d-ecefaa3a53ed None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:42.813958 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bfab0c-9a17-42dd-ba2d-ecefaa3a53ed None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:42.836582 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908d8adb-c3c3-43a2-8908-73817a0a61b7 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:42.838893 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:42.865640 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66ef292-2059-40a7-a959-aac8de640327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:42.866257 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ef292-2059-40a7-a959-aac8de640327 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:42.866546 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ef292-2059-40a7-a959-aac8de640327 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:42.866819 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ef292-2059-40a7-a959-aac8de640327 None admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:42.867565 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66ef292-2059-40a7-a959-aac8de640327 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:42.867875 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66ef292-2059-40a7-a959-aac8de640327 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:42.869310 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66ef292-2059-40a7-a959-aac8de640327 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:42.875774 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460a1d87-3d37-4c6a-aadb-bb2b63af3d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:42.876473 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460a1d87-3d37-4c6a-aadb-bb2b63af3d35 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:42.876819 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460a1d87-3d37-4c6a-aadb-bb2b63af3d35 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:42.877154 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460a1d87-3d37-4c6a-aadb-bb2b63af3d35 None admin] PATH_INFO: `/v3/domains` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:42.878501 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460a1d87-3d37-4c6a-aadb-bb2b63af3d35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:42.878867 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460a1d87-3d37-4c6a-aadb-bb2b63af3d35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:42.880748 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460a1d87-3d37-4c6a-aadb-bb2b63af3d35 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:42.888343 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 317/644] 10.222.0.58 () {56 vars in 1148 bytes} [Tue Jul 26 13:33:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:42.888343 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66ef292-2059-40a7-a959-aac8de640327 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:42.903798 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 328/645] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Jul 26 13:33:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:42.903798 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460a1d87-3d37-4c6a-aadb-bb2b63af3d35 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:42.912364 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:43.017068 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66ef292-2059-40a7-a959-aac8de640327 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:43.041271 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460a1d87-3d37-4c6a-aadb-bb2b63af3d35 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:43.063131 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256dbb08-f250-4fa7-a734-86d062985f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:43.064258 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256dbb08-f250-4fa7-a734-86d062985f8d None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:43.064551 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256dbb08-f250-4fa7-a734-86d062985f8d None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:43.064820 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256dbb08-f250-4fa7-a734-86d062985f8d None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:43.065579 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256dbb08-f250-4fa7-a734-86d062985f8d None admin] RBAC: Adding query filter params (name=service) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:43.065886 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256dbb08-f250-4fa7-a734-86d062985f8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:43.067273 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256dbb08-f250-4fa7-a734-86d062985f8d None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:43.076619 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 318/646] 10.222.0.58 () {56 vars in 1103 bytes} [Tue Jul 26 13:33:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:43.077453 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256dbb08-f250-4fa7-a734-86d062985f8d None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:43.085626 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba47f329-5701-426b-b8f2-a5e3fa19dc70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:43.086356 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba47f329-5701-426b-b8f2-a5e3fa19dc70 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:43.086688 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba47f329-5701-426b-b8f2-a5e3fa19dc70 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:43.086919 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba47f329-5701-426b-b8f2-a5e3fa19dc70 None admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:43.088198 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba47f329-5701-426b-b8f2-a5e3fa19dc70 None admin] RBAC: Adding query filter params (name=service) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:43.088507 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba47f329-5701-426b-b8f2-a5e3fa19dc70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:43.089913 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba47f329-5701-426b-b8f2-a5e3fa19dc70 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:43.099448 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 329/647] 10.222.0.58 () {56 vars in 1103 bytes} [Tue Jul 26 13:33:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:43.100104 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba47f329-5701-426b-b8f2-a5e3fa19dc70 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:43.136373 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:43.183223 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:43.211729 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256dbb08-f250-4fa7-a734-86d062985f8d None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:43.224161 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba47f329-5701-426b-b8f2-a5e3fa19dc70 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:43.250113 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076dbafc-eac5-47fa-823d-7720875be1dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:43.250733 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076dbafc-eac5-47fa-823d-7720875be1dc None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:43.251017 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076dbafc-eac5-47fa-823d-7720875be1dc None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:43.251289 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076dbafc-eac5-47fa-823d-7720875be1dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:43.252076 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076dbafc-eac5-47fa-823d-7720875be1dc None admin] RBAC: Adding query filter params (role.id=17e0f3c73a8a4a54b3e7b78878dfbf09, scope.project.id=1e8bb4385c204210a1bc77b1f0d654f3, user.id=c69161cd520b4b29bf50f3bf2f1c033b) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:43.252469 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076dbafc-eac5-47fa-823d-7720875be1dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:43.253854 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076dbafc-eac5-47fa-823d-7720875be1dc None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:43.260881 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 319/648] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Jul 26 13:33:43 2022] GET /identity/v3/role_assignments?user.id=c69161cd520b4b29bf50f3bf2f1c033b&scope.project.id=1e8bb4385c204210a1bc77b1f0d654f3&role.id=17e0f3c73a8a4a54b3e7b78878dfbf09 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:43.261435 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076dbafc-eac5-47fa-823d-7720875be1dc None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:43.275598 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0bf27f-5fdc-4f8f-8608-4d7d127a184f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:43.277576 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0bf27f-5fdc-4f8f-8608-4d7d127a184f None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:43.278127 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0bf27f-5fdc-4f8f-8608-4d7d127a184f None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:43.278648 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0bf27f-5fdc-4f8f-8608-4d7d127a184f None admin] PATH_INFO: `/v3/users/glance` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:43.291191 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0bf27f-5fdc-4f8f-8608-4d7d127a184f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:43.294263 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0bf27f-5fdc-4f8f-8608-4d7d127a184f None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:43.311869 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-3f0bf27f-5fdc-4f8f-8608-4d7d127a184f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 26 13:33:43.311869 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:43.311869 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:43.311869 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:43.311869 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:43.311869 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:43.311869 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:43.311869 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:43.311869 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:43.311869 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:43.311869 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:43.311869 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:43.311869 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 13:33:43.311869 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 13:33:43.311869 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 13:33:43.311869 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 13:33:43.312955 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:43.312955 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:43.312955 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 13:33:43.312955 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 13:33:43.312955 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 13:33:43.312955 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 13:33:43.312955 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:43.312955 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:43.312955 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:43.312955 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:43.312955 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:43.312955 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:43.312955 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:43.312955 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:43.312955 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:43.312955 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:43.312955 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:43.312955 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:43.313943 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:43.313943 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:43.313943 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 13:33:43.313943 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 13:33:43.313943 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 13:33:43.313943 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 13:33:43.313943 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 13:33:43.313943 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 13:33:43.313943 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 26 13:33:43.313943 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:33:43.319738 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 330/649] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Jul 26 13:33:43 2022] GET /identity/v3/users/glance => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:43.321847 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0bf27f-5fdc-4f8f-8608-4d7d127a184f None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:43.411490 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076dbafc-eac5-47fa-823d-7720875be1dc None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:43.416692 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:43.459467 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0bf27f-5fdc-4f8f-8608-4d7d127a184f None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:43.467516 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb319729-b586-46b8-af67-3d552bb6cd9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:43.468180 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb319729-b586-46b8-af67-3d552bb6cd9d None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:43.468462 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb319729-b586-46b8-af67-3d552bb6cd9d None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:43.468734 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb319729-b586-46b8-af67-3d552bb6cd9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:43.481590 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb319729-b586-46b8-af67-3d552bb6cd9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:43.483048 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb319729-b586-46b8-af67-3d552bb6cd9d None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:43.496096 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:43.502896 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-bb319729-b586-46b8-af67-3d552bb6cd9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:43.502896 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:43.502896 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:43.502896 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:43.502896 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:43.502896 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:43.502896 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:43.502896 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:43.502896 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:43.502896 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:43.502896 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:43.502896 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:43.502896 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 13:33:43.502896 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 13:33:43.502896 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 13:33:43.504424 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 13:33:43.504424 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:43.504424 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:43.504424 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:43.504424 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:43.504424 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:43.504424 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:43.504424 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:43.504424 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:43.504424 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:43.504424 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:43.504424 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:43.504424 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:43.504424 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:43.504424 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:43.504424 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:43.504424 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:43.506025 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 13:33:43.506025 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 13:33:43.506025 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:43.506025 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:33:43.506025 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 320/650] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:33:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:43.506025 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb319729-b586-46b8-af67-3d552bb6cd9d None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:43.510086 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274eed57-cfc1-4348-92da-f397cca1c9a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:43.510748 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274eed57-cfc1-4348-92da-f397cca1c9a1 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:43.511024 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274eed57-cfc1-4348-92da-f397cca1c9a1 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:43.511295 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274eed57-cfc1-4348-92da-f397cca1c9a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:43.525690 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274eed57-cfc1-4348-92da-f397cca1c9a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:43.527141 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274eed57-cfc1-4348-92da-f397cca1c9a1 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:43.543750 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-274eed57-cfc1-4348-92da-f397cca1c9a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:43.543750 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:43.543750 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:43.543750 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:43.543750 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:43.543750 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:43.543750 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:43.543750 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:43.543750 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:43.543750 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:43.543750 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:43.543750 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:43.543750 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 13:33:43.543750 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 13:33:43.543750 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 13:33:43.544702 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 13:33:43.544702 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:43.544702 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:43.544702 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:43.544702 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:43.544702 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:43.544702 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:43.544702 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:43.544702 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:43.544702 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:43.544702 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:43.544702 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:43.544702 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:43.544702 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:43.544702 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:43.544702 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:43.544702 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:43.546032 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 13:33:43.546032 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 13:33:43.546032 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:43.546032 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:33:43.546032 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 331/651] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:33:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:43.550268 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274eed57-cfc1-4348-92da-f397cca1c9a1 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:43.650819 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb319729-b586-46b8-af67-3d552bb6cd9d None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:43.700108 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274eed57-cfc1-4348-92da-f397cca1c9a1 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:43.705178 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4007d14-f368-4e2f-94ea-7de761cb2a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:43.706241 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4007d14-f368-4e2f-94ea-7de761cb2a54 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:43.706737 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4007d14-f368-4e2f-94ea-7de761cb2a54 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:43.707212 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4007d14-f368-4e2f-94ea-7de761cb2a54 None admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:43.708486 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4007d14-f368-4e2f-94ea-7de761cb2a54 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:43.708993 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4007d14-f368-4e2f-94ea-7de761cb2a54 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:43.712806 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4007d14-f368-4e2f-94ea-7de761cb2a54 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:43.740165 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 321/652] 10.222.0.58 () {56 vars in 1137 bytes} [Tue Jul 26 13:33:43 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:43.741575 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4007d14-f368-4e2f-94ea-7de761cb2a54 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:43.750710 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3b1ce3-ee34-40a8-ba88-fff311de8644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:43.751680 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3b1ce3-ee34-40a8-ba88-fff311de8644 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:43.752285 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3b1ce3-ee34-40a8-ba88-fff311de8644 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:43.752980 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3b1ce3-ee34-40a8-ba88-fff311de8644 None admin] PATH_INFO: `/v3/domains` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:43.754462 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3b1ce3-ee34-40a8-ba88-fff311de8644 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:43.755232 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3b1ce3-ee34-40a8-ba88-fff311de8644 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:43.757720 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3b1ce3-ee34-40a8-ba88-fff311de8644 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:43.761073 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:43.778771 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 332/653] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Jul 26 13:33:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:43.780052 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3b1ce3-ee34-40a8-ba88-fff311de8644 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:43.840237 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:43.864646 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4007d14-f368-4e2f-94ea-7de761cb2a54 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:43.917217 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472d8680-6556-4803-9f2d-98faaff7ec11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:43.917930 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472d8680-6556-4803-9f2d-98faaff7ec11 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:43.918358 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472d8680-6556-4803-9f2d-98faaff7ec11 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:43.918637 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472d8680-6556-4803-9f2d-98faaff7ec11 None admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:43.919349 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472d8680-6556-4803-9f2d-98faaff7ec11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:43.919657 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472d8680-6556-4803-9f2d-98faaff7ec11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:43.921402 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472d8680-6556-4803-9f2d-98faaff7ec11 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:43.931833 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3b1ce3-ee34-40a8-ba88-fff311de8644 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:43.938021 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 322/654] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Jul 26 13:33:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:43.938901 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472d8680-6556-4803-9f2d-98faaff7ec11 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:43.974848 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ddfbd0-9fe7-46ec-ae89-54588b6ece6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:43.976076 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ddfbd0-9fe7-46ec-ae89-54588b6ece6e None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:43.976568 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ddfbd0-9fe7-46ec-ae89-54588b6ece6e None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:43.977103 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ddfbd0-9fe7-46ec-ae89-54588b6ece6e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:44.041753 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:44.072850 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ddfbd0-9fe7-46ec-ae89-54588b6ece6e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:44.074499 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ddfbd0-9fe7-46ec-ae89-54588b6ece6e None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:44.080566 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472d8680-6556-4803-9f2d-98faaff7ec11 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:44.119196 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da74e2c6-a0c8-4d5d-b22c-e1801f99be6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:44.119854 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da74e2c6-a0c8-4d5d-b22c-e1801f99be6f None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:44.120134 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da74e2c6-a0c8-4d5d-b22c-e1801f99be6f None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:44.120399 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da74e2c6-a0c8-4d5d-b22c-e1801f99be6f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:44.129167 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:44.131340 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da74e2c6-a0c8-4d5d-b22c-e1801f99be6f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:44.133140 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da74e2c6-a0c8-4d5d-b22c-e1801f99be6f None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:44.145749 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-da74e2c6-a0c8-4d5d-b22c-e1801f99be6f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 26 13:33:44.145749 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:44.145749 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:44.145749 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:44.145749 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:44.145749 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:44.145749 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:44.145749 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:44.145749 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:44.145749 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:44.145749 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:44.145749 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:44.145749 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 13:33:44.145749 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 13:33:44.145749 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 13:33:44.145749 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 13:33:44.146746 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:44.146746 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:44.146746 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 13:33:44.146746 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 13:33:44.146746 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 13:33:44.146746 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 13:33:44.146746 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:44.146746 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:44.146746 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:44.146746 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:44.146746 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:44.146746 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:44.146746 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:44.146746 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:44.146746 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:44.146746 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:44.146746 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:44.146746 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:44.147668 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:44.147668 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:44.147668 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 13:33:44.147668 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 13:33:44.147668 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 13:33:44.147668 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 13:33:44.147668 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 13:33:44.147668 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 13:33:44.147668 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 26 13:33:44.147668 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:33:44.151576 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 323/655] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Jul 26 13:33:43 2022] GET /identity/v3/users/cinder => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:44.152506 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da74e2c6-a0c8-4d5d-b22c-e1801f99be6f None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:44.209745 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 333/656] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Jul 26 13:33:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 13:33:44.209745 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d837514e-070b-45a8-a088-4e70be2336d1 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:44.209745 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d837514e-070b-45a8-a088-4e70be2336d1 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:44.209745 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d837514e-070b-45a8-a088-4e70be2336d1 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:44.209745 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 334/657] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:33:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:33:44.209745 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d837514e-070b-45a8-a088-4e70be2336d1 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:44.288290 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da74e2c6-a0c8-4d5d-b22c-e1801f99be6f None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:44.325112 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d837514e-070b-45a8-a088-4e70be2336d1 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:44.325775 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6310b0-f183-4dc1-8dd0-793afe3294b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:44.326429 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6310b0-f183-4dc1-8dd0-793afe3294b4 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:44.326712 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6310b0-f183-4dc1-8dd0-793afe3294b4 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:44.327024 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6310b0-f183-4dc1-8dd0-793afe3294b4 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:44.329053 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:44.336209 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6310b0-f183-4dc1-8dd0-793afe3294b4 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:44.339330 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6310b0-f183-4dc1-8dd0-793afe3294b4 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:44.351898 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-5b6310b0-f183-4dc1-8dd0-793afe3294b4 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 26 13:33:44.351898 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:44.351898 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:44.351898 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:44.351898 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:44.351898 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:44.351898 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:44.351898 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:44.351898 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:44.351898 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:44.351898 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:44.351898 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:44.351898 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 13:33:44.351898 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 13:33:44.351898 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 13:33:44.351898 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 13:33:44.353753 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:44.353753 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:44.353753 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 13:33:44.353753 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 13:33:44.353753 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 13:33:44.353753 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 13:33:44.353753 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:44.353753 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:44.353753 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:44.353753 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:44.353753 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:44.353753 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:44.353753 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:44.353753 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:44.353753 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:44.353753 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:44.353753 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:44.353753 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:44.354902 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:44.354902 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:44.354902 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 13:33:44.354902 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 13:33:44.354902 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 13:33:44.354902 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 13:33:44.354902 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 13:33:44.354902 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 13:33:44.354902 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 26 13:33:44.354902 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:33:44.358374 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 324/658] 10.222.0.58 () {56 vars in 1102 bytes} [Tue Jul 26 13:33:44 2022] GET /identity/v3/users/neutron => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:44.359732 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202f500b-7704-4b58-82ea-cba71e5dfaea None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:44.360023 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202f500b-7704-4b58-82ea-cba71e5dfaea None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:44.360307 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202f500b-7704-4b58-82ea-cba71e5dfaea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:44.376110 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c45c9b-032b-443a-a38d-8891371636f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:44.376972 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c45c9b-032b-443a-a38d-8891371636f3 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:44.377395 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c45c9b-032b-443a-a38d-8891371636f3 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:44.377862 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c45c9b-032b-443a-a38d-8891371636f3 None admin] PATH_INFO: `/v3/users` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:44.378485 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-202f500b-7704-4b58-82ea-cba71e5dfaea None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:33:44.378968 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c45c9b-032b-443a-a38d-8891371636f3 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:44.379416 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c45c9b-032b-443a-a38d-8891371636f3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:44.379705 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-202f500b-7704-4b58-82ea-cba71e5dfaea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:33:44.381211 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c45c9b-032b-443a-a38d-8891371636f3 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:44.395005 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:44.411713 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 335/659] 10.222.0.58 () {56 vars in 1137 bytes} [Tue Jul 26 13:33:44 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:44.411713 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c45c9b-032b-443a-a38d-8891371636f3 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:44.469259 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 325/660] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:33:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 13:33:44.470229 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202f500b-7704-4b58-82ea-cba71e5dfaea None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:44.555042 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c45c9b-032b-443a-a38d-8891371636f3 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:44.597142 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202f500b-7704-4b58-82ea-cba71e5dfaea None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:44.597536 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31c40eb-8af7-49e7-a30f-4a69c9b7ec64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:44.598202 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31c40eb-8af7-49e7-a30f-4a69c9b7ec64 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:44.598490 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31c40eb-8af7-49e7-a30f-4a69c9b7ec64 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:44.598779 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31c40eb-8af7-49e7-a30f-4a69c9b7ec64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:44.616535 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31c40eb-8af7-49e7-a30f-4a69c9b7ec64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:44.618049 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31c40eb-8af7-49e7-a30f-4a69c9b7ec64 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:44.623282 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:44.635742 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-c31c40eb-8af7-49e7-a30f-4a69c9b7ec64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:44.635742 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:44.635742 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:44.635742 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:44.635742 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:44.635742 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:44.635742 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:44.635742 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:44.635742 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:44.635742 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:44.635742 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:44.635742 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:44.635742 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 13:33:44.635742 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 13:33:44.635742 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 13:33:44.636705 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 13:33:44.636705 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:44.636705 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:44.636705 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:44.636705 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:44.636705 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:44.636705 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:44.636705 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:44.636705 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:44.636705 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:44.636705 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:44.636705 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:44.636705 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:44.636705 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:44.636705 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:44.636705 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:44.636705 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:44.637642 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 13:33:44.637642 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 13:33:44.637642 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:44.637642 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:33:44.637642 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 336/661] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:33:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:44.637917 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31c40eb-8af7-49e7-a30f-4a69c9b7ec64 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:44.649134 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350da610-a3ac-4629-8268-655c406a4fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:44.651096 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350da610-a3ac-4629-8268-655c406a4fd0 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:44.651556 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350da610-a3ac-4629-8268-655c406a4fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:44.652314 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350da610-a3ac-4629-8268-655c406a4fd0 None admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:44.653615 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350da610-a3ac-4629-8268-655c406a4fd0 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:44.654102 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350da610-a3ac-4629-8268-655c406a4fd0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:44.656407 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350da610-a3ac-4629-8268-655c406a4fd0 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:44.683993 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 326/662] 10.222.0.58 () {56 vars in 1139 bytes} [Tue Jul 26 13:33:44 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:44.683993 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350da610-a3ac-4629-8268-655c406a4fd0 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:44.695901 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:44.755781 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31c40eb-8af7-49e7-a30f-4a69c9b7ec64 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:44.799082 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9f0ee1-569e-40c8-9ee4-587db6429f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:44.800074 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9f0ee1-569e-40c8-9ee4-587db6429f93 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:44.800559 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9f0ee1-569e-40c8-9ee4-587db6429f93 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:44.801049 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9f0ee1-569e-40c8-9ee4-587db6429f93 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:44.806867 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350da610-a3ac-4629-8268-655c406a4fd0 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:44.846511 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be280fb5-976a-470b-9fa2-1635d7b2605f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:44.847535 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be280fb5-976a-470b-9fa2-1635d7b2605f None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:44.848001 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be280fb5-976a-470b-9fa2-1635d7b2605f None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:44.848369 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be280fb5-976a-470b-9fa2-1635d7b2605f None admin] PATH_INFO: `/v3/roles/service` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:44.862044 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be280fb5-976a-470b-9fa2-1635d7b2605f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:33:44.866907 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-be280fb5-976a-470b-9fa2-1635d7b2605f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 13:33:44.866907 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:44.866907 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:44.866907 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:44.866907 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:44.866907 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:44.866907 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:44.866907 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:44.866907 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:44.866907 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:44.866907 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:44.866907 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:44.866907 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:33:44.866907 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:33:44.866907 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:33:44.866907 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:33:44.867933 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:33:44.867933 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:33:44.867933 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:33:44.867933 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:33:44.867933 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:44.867933 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:44.867933 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:44.867933 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:44.867933 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:44.867933 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:44.867933 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:44.867933 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:44.867933 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:44.867933 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:44.867933 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:44.867933 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:44.868811 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:44.868811 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:44.868811 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:44.868811 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:44.868811 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:33:44.868811 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:33:44.868811 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:33:44.868811 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:33:44.868811 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:33:44.868811 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:33:44.868811 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 13:33:44.868811 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:33:44.868811 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 327/663] 10.222.0.58 () {56 vars in 1102 bytes} [Tue Jul 26 13:33:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:44.869512 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be280fb5-976a-470b-9fa2-1635d7b2605f None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:44.896356 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9f0ee1-569e-40c8-9ee4-587db6429f93 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:44.898245 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9f0ee1-569e-40c8-9ee4-587db6429f93 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:44.932244 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:44.974456 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:45.007558 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be280fb5-976a-470b-9fa2-1635d7b2605f None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:45.049061 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4dae49-827d-4d8c-bb98-fa768e396eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:45.049829 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4dae49-827d-4d8c-bb98-fa768e396eac None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:45.050176 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4dae49-827d-4d8c-bb98-fa768e396eac None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:45.050520 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4dae49-827d-4d8c-bb98-fa768e396eac None admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:45.051299 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4dae49-827d-4d8c-bb98-fa768e396eac None admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:45.051673 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4dae49-827d-4d8c-bb98-fa768e396eac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:45.052758 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 337/664] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Jul 26 13:33:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 13:33:45.053683 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9f0ee1-569e-40c8-9ee4-587db6429f93 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:45.058137 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4dae49-827d-4d8c-bb98-fa768e396eac None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:45.071953 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 328/665] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Jul 26 13:33:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:45.072870 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4dae49-827d-4d8c-bb98-fa768e396eac None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:45.180478 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9f0ee1-569e-40c8-9ee4-587db6429f93 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:45.182077 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4dae49-827d-4d8c-bb98-fa768e396eac None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:45.183674 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:45.225949 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba71ca5d-eb81-4d46-9d42-869253946828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:45.226246 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3078fc3-fce1-4f08-99e5-463d809182d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:45.226597 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3078fc3-fce1-4f08-99e5-463d809182d1 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:45.226737 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba71ca5d-eb81-4d46-9d42-869253946828 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:45.226872 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba71ca5d-eb81-4d46-9d42-869253946828 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:45.226986 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3078fc3-fce1-4f08-99e5-463d809182d1 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:45.227098 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba71ca5d-eb81-4d46-9d42-869253946828 None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:45.227217 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3078fc3-fce1-4f08-99e5-463d809182d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:45.227873 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba71ca5d-eb81-4d46-9d42-869253946828 None admin] RBAC: Adding query filter params (name=service) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:45.228197 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba71ca5d-eb81-4d46-9d42-869253946828 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:45.229602 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba71ca5d-eb81-4d46-9d42-869253946828 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:45.242496 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 329/666] 10.222.0.58 () {56 vars in 1103 bytes} [Tue Jul 26 13:33:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:45.242496 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba71ca5d-eb81-4d46-9d42-869253946828 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:45.257661 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:45.321605 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3078fc3-fce1-4f08-99e5-463d809182d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:45.323218 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3078fc3-fce1-4f08-99e5-463d809182d1 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:45.364523 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba71ca5d-eb81-4d46-9d42-869253946828 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:45.401945 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8e3696-3500-4985-8b1e-16e14fc93420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:45.402600 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8e3696-3500-4985-8b1e-16e14fc93420 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:45.402885 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8e3696-3500-4985-8b1e-16e14fc93420 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:45.403282 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8e3696-3500-4985-8b1e-16e14fc93420 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:45.416293 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8e3696-3500-4985-8b1e-16e14fc93420 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:45.417910 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8e3696-3500-4985-8b1e-16e14fc93420 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:45.435318 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-3a8e3696-3500-4985-8b1e-16e14fc93420 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:45.435318 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:45.435318 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:45.435318 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:45.435318 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:45.435318 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:45.435318 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:45.435318 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:45.435318 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:45.435318 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:45.435318 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:45.435318 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:45.435318 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 13:33:45.435318 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 13:33:45.435318 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 13:33:45.441980 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 13:33:45.441980 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:45.441980 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:45.441980 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:45.441980 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:45.441980 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:45.441980 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:45.441980 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:45.441980 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:45.441980 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:45.441980 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:45.441980 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:45.441980 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:45.441980 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:45.441980 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:45.441980 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:45.441980 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:45.443765 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 13:33:45.443765 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 13:33:45.443765 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:45.443765 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:33:45.443765 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 330/667] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:33:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:45.443765 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8e3696-3500-4985-8b1e-16e14fc93420 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:45.444365 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 338/668] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Jul 26 13:33:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 13:33:45.444365 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3078fc3-fce1-4f08-99e5-463d809182d1 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:45.461706 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:45.495440 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:45.546982 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8e3696-3500-4985-8b1e-16e14fc93420 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:45.553694 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3078fc3-fce1-4f08-99e5-463d809182d1 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:45.584613 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed00af05-20b8-49e5-96c2-ab06d0cef653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:45.585260 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed00af05-20b8-49e5-96c2-ab06d0cef653 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:45.585566 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed00af05-20b8-49e5-96c2-ab06d0cef653 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:45.585838 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed00af05-20b8-49e5-96c2-ab06d0cef653 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:45.592074 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45bb8d9-c079-41a2-b7a3-948d0b145722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:45.592655 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed00af05-20b8-49e5-96c2-ab06d0cef653 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:45.592762 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45bb8d9-c079-41a2-b7a3-948d0b145722 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:45.592991 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45bb8d9-c079-41a2-b7a3-948d0b145722 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:45.593283 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45bb8d9-c079-41a2-b7a3-948d0b145722 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:45.594142 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed00af05-20b8-49e5-96c2-ab06d0cef653 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:45.604561 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45bb8d9-c079-41a2-b7a3-948d0b145722 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:45.606092 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45bb8d9-c079-41a2-b7a3-948d0b145722 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:45.606450 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-ed00af05-20b8-49e5-96c2-ab06d0cef653 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 13:33:45.606450 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:45.606450 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:45.606450 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:45.606450 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:45.606450 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:45.606450 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:45.606450 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:45.606450 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:45.606450 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:45.606450 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:45.606450 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:45.606450 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 13:33:45.606450 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 13:33:45.606450 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 13:33:45.607336 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 13:33:45.607336 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:45.607336 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:45.607336 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:45.607336 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:45.607336 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:45.607336 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:45.607336 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:45.607336 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:45.607336 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:45.607336 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:45.607336 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:45.607336 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:45.607336 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:45.607336 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:45.607336 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:45.607336 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:45.608218 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 13:33:45.608218 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 13:33:45.608218 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 13:33:45.608218 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 13:33:45.608218 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 13:33:45.608218 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 13:33:45.608218 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 13:33:45.608218 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:33:45.608218 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 331/669] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Jul 26 13:33:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:45.608703 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed00af05-20b8-49e5-96c2-ab06d0cef653 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:45.626764 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-d45bb8d9-c079-41a2-b7a3-948d0b145722 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:45.626764 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:45.626764 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:45.626764 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:45.626764 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:45.626764 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:45.626764 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:45.626764 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:45.626764 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:45.626764 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:45.626764 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:45.626764 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:45.626764 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 13:33:45.626764 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 13:33:45.626764 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 13:33:45.627875 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 13:33:45.627875 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:45.627875 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:45.627875 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:45.627875 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:45.627875 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:45.627875 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:45.627875 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:45.627875 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:45.627875 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:45.627875 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:45.627875 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:45.627875 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:45.627875 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:45.627875 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:45.627875 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:45.627875 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:45.628747 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 13:33:45.628747 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 13:33:45.628747 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:45.628747 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:33:45.628747 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 339/670] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:33:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:45.629529 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8398e0e5-126d-4cb9-9b9b-4512cbab82c2 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:45.629633 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8398e0e5-126d-4cb9-9b9b-4512cbab82c2 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:45.629909 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8398e0e5-126d-4cb9-9b9b-4512cbab82c2 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:45.631348 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 340/671] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:33:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:33:45.631966 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8398e0e5-126d-4cb9-9b9b-4512cbab82c2 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:45.697577 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:45.722427 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed00af05-20b8-49e5-96c2-ab06d0cef653 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:45.740020 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8398e0e5-126d-4cb9-9b9b-4512cbab82c2 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:45.741681 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:45.757530 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3b0683-2399-4cf9-9cc6-149bf83a2863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:45.758175 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3b0683-2399-4cf9-9cc6-149bf83a2863 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:45.758520 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3b0683-2399-4cf9-9cc6-149bf83a2863 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:45.758835 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3b0683-2399-4cf9-9cc6-149bf83a2863 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:45.775043 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e34bcc-9b34-4072-a2a3-c960ed06eb20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:45.775665 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e34bcc-9b34-4072-a2a3-c960ed06eb20 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:45.775941 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e34bcc-9b34-4072-a2a3-c960ed06eb20 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:45.776213 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e34bcc-9b34-4072-a2a3-c960ed06eb20 None admin] PATH_INFO: `/v3/domains` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:45.776355 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3b0683-2399-4cf9-9cc6-149bf83a2863 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:45.776907 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e34bcc-9b34-4072-a2a3-c960ed06eb20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:45.777215 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e34bcc-9b34-4072-a2a3-c960ed06eb20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:45.777864 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3b0683-2399-4cf9-9cc6-149bf83a2863 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:45.778613 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e34bcc-9b34-4072-a2a3-c960ed06eb20 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:45.791303 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 341/672] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Jul 26 13:33:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:45.792145 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e34bcc-9b34-4072-a2a3-c960ed06eb20 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:45.793952 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-0d3b0683-2399-4cf9-9cc6-149bf83a2863 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:45.793952 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:45.793952 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:45.793952 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:45.793952 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:45.793952 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:45.793952 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:45.793952 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:45.793952 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:45.793952 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:45.793952 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:45.793952 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:45.793952 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 13:33:45.793952 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 13:33:45.793952 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 13:33:45.794811 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 13:33:45.794811 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:45.794811 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:45.794811 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:45.794811 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:45.794811 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:45.794811 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:45.794811 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:45.794811 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:45.794811 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:45.794811 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:45.794811 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:45.794811 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:45.794811 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:45.794811 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:45.794811 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:45.794811 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:45.795740 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 13:33:45.795740 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 13:33:45.795740 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:45.795740 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:33:45.795740 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 332/673] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:33:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:45.796579 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cab47ed-00a5-4653-885f-8e217a442a98 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:45.796866 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cab47ed-00a5-4653-885f-8e217a442a98 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:45.797135 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cab47ed-00a5-4653-885f-8e217a442a98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:45.820509 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-1cab47ed-00a5-4653-885f-8e217a442a98 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:33:45.821987 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-1cab47ed-00a5-4653-885f-8e217a442a98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:33:45.898721 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e34bcc-9b34-4072-a2a3-c960ed06eb20 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:45.901192 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 333/674] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:33:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 13:33:45.901861 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cab47ed-00a5-4653-885f-8e217a442a98 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:45.943692 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8956364a-fcc1-4a7f-bdc0-9c5555ef3cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:45.944328 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8956364a-fcc1-4a7f-bdc0-9c5555ef3cde None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:45.944607 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8956364a-fcc1-4a7f-bdc0-9c5555ef3cde None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:45.944884 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8956364a-fcc1-4a7f-bdc0-9c5555ef3cde None admin] PATH_INFO: `/v3/projects` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:45.945670 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8956364a-fcc1-4a7f-bdc0-9c5555ef3cde None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:45.945983 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8956364a-fcc1-4a7f-bdc0-9c5555ef3cde None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:45.947424 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8956364a-fcc1-4a7f-bdc0-9c5555ef3cde None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:45.963026 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 342/675] 10.222.0.58 () {56 vars in 1148 bytes} [Tue Jul 26 13:33:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:45.963880 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8956364a-fcc1-4a7f-bdc0-9c5555ef3cde None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:45.974072 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:45.990650 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:46.021932 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cab47ed-00a5-4653-885f-8e217a442a98 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:46.062596 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1920d56d-6694-4d7c-80bb-1bb31a2027d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:46.063224 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1920d56d-6694-4d7c-80bb-1bb31a2027d9 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:46.063509 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1920d56d-6694-4d7c-80bb-1bb31a2027d9 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:46.063786 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1920d56d-6694-4d7c-80bb-1bb31a2027d9 None admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:46.064492 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1920d56d-6694-4d7c-80bb-1bb31a2027d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:46.064797 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1920d56d-6694-4d7c-80bb-1bb31a2027d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:46.066221 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1920d56d-6694-4d7c-80bb-1bb31a2027d9 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:46.074025 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8956364a-fcc1-4a7f-bdc0-9c5555ef3cde None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:46.078408 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 334/676] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Jul 26 13:33:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:46.079244 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1920d56d-6694-4d7c-80bb-1bb31a2027d9 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:46.115313 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab9f472-c031-4c4f-a06e-d9dba4294ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:46.115976 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab9f472-c031-4c4f-a06e-d9dba4294ff0 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:46.116250 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab9f472-c031-4c4f-a06e-d9dba4294ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:46.116519 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab9f472-c031-4c4f-a06e-d9dba4294ff0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:46.123836 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab9f472-c031-4c4f-a06e-d9dba4294ff0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:46.125275 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab9f472-c031-4c4f-a06e-d9dba4294ff0 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:46.137608 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-2ab9f472-c031-4c4f-a06e-d9dba4294ff0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 13:33:46.137608 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:46.137608 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:46.137608 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:46.137608 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:46.137608 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:46.137608 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:46.137608 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:46.137608 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:46.137608 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:46.137608 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:46.137608 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:46.137608 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 13:33:46.137608 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 13:33:46.137608 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 13:33:46.138654 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 13:33:46.138654 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:46.138654 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:46.138654 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:46.138654 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:46.138654 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:46.138654 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:46.138654 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:46.138654 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:46.138654 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:46.138654 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:46.138654 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:46.138654 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:46.138654 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:46.138654 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:46.138654 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:46.138654 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:46.139531 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 13:33:46.139531 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 13:33:46.139531 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 13:33:46.139531 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 13:33:46.139531 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 13:33:46.139531 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 13:33:46.139531 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 13:33:46.139531 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:33:46.139531 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 343/677] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Jul 26 13:33:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:46.139531 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab9f472-c031-4c4f-a06e-d9dba4294ff0 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:46.198954 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1920d56d-6694-4d7c-80bb-1bb31a2027d9 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:46.216354 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:46.225582 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:46.243057 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed90201-8ec8-4be4-8161-faf530f73ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:46.243695 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed90201-8ec8-4be4-8161-faf530f73ae9 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:46.243976 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed90201-8ec8-4be4-8161-faf530f73ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:46.244250 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed90201-8ec8-4be4-8161-faf530f73ae9 None admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:46.244514 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab9f472-c031-4c4f-a06e-d9dba4294ff0 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:46.244949 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed90201-8ec8-4be4-8161-faf530f73ae9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:46.245253 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed90201-8ec8-4be4-8161-faf530f73ae9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:46.246663 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed90201-8ec8-4be4-8161-faf530f73ae9 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:46.259633 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 335/678] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Jul 26 13:33:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:46.260486 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed90201-8ec8-4be4-8161-faf530f73ae9 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:46.280673 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114fd4ee-9597-485a-87f2-b8c433c44ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:46.281321 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114fd4ee-9597-485a-87f2-b8c433c44ff4 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:46.281803 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114fd4ee-9597-485a-87f2-b8c433c44ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:46.282089 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114fd4ee-9597-485a-87f2-b8c433c44ff4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:46.292895 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114fd4ee-9597-485a-87f2-b8c433c44ff4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:33:46.298617 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-114fd4ee-9597-485a-87f2-b8c433c44ff4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 13:33:46.298617 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:46.298617 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:46.298617 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:46.298617 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:46.298617 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:46.298617 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:46.298617 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:46.298617 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:46.298617 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:46.298617 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:46.298617 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:46.298617 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:33:46.298617 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:33:46.298617 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:33:46.298617 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:33:46.299610 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:33:46.299610 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:33:46.299610 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:33:46.299610 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:33:46.299610 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:46.299610 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:46.299610 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:46.299610 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:46.299610 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:46.299610 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:46.299610 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:46.299610 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:46.299610 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:46.299610 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:46.299610 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:46.299610 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:46.300462 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:46.300462 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:46.300462 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:46.300462 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:46.300462 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:33:46.300462 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:33:46.300462 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:33:46.300462 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:33:46.300462 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:33:46.300462 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:33:46.300462 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 13:33:46.300462 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:33:46.300462 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 344/679] 10.222.0.58 () {56 vars in 1096 bytes} [Tue Jul 26 13:33:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:46.301331 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114fd4ee-9597-485a-87f2-b8c433c44ff4 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:46.363534 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed90201-8ec8-4be4-8161-faf530f73ae9 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:46.398631 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c4024d-16df-4ebb-98c6-170b851dc5a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:46.399245 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c4024d-16df-4ebb-98c6-170b851dc5a6 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:46.400082 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c4024d-16df-4ebb-98c6-170b851dc5a6 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:46.400384 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c4024d-16df-4ebb-98c6-170b851dc5a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:46.401183 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c4024d-16df-4ebb-98c6-170b851dc5a6 None admin] RBAC: Adding query filter params (role.id=17e0f3c73a8a4a54b3e7b78878dfbf09, scope.project.id=1e8bb4385c204210a1bc77b1f0d654f3, user.id=eaf065aef9a046b2adaec34816a336b7) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:46.401517 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c4024d-16df-4ebb-98c6-170b851dc5a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:46.402963 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c4024d-16df-4ebb-98c6-170b851dc5a6 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:46.409191 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 336/680] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Jul 26 13:33:46 2022] GET /identity/v3/role_assignments?user.id=eaf065aef9a046b2adaec34816a336b7&scope.project.id=1e8bb4385c204210a1bc77b1f0d654f3&role.id=17e0f3c73a8a4a54b3e7b78878dfbf09 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:46.410042 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c4024d-16df-4ebb-98c6-170b851dc5a6 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:46.415747 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114fd4ee-9597-485a-87f2-b8c433c44ff4 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:46.462730 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fd04dd-9184-4b83-bdef-1342d2fda64b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:46.463307 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fd04dd-9184-4b83-bdef-1342d2fda64b None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:46.463600 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fd04dd-9184-4b83-bdef-1342d2fda64b None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:46.463873 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fd04dd-9184-4b83-bdef-1342d2fda64b None admin] PATH_INFO: `/v3/users/swift` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:46.465881 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:46.474662 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fd04dd-9184-4b83-bdef-1342d2fda64b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:46.476139 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fd04dd-9184-4b83-bdef-1342d2fda64b None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:46.490029 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-30fd04dd-9184-4b83-bdef-1342d2fda64b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 26 13:33:46.490029 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:46.490029 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:46.490029 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:46.490029 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:46.490029 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:46.490029 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:46.490029 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:46.490029 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:46.490029 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:46.490029 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:46.490029 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:46.490029 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 13:33:46.490029 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 13:33:46.490029 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 13:33:46.490029 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 13:33:46.492005 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:46.492005 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:46.492005 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 13:33:46.492005 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 13:33:46.492005 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 13:33:46.492005 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 13:33:46.492005 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:46.492005 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:46.492005 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:46.492005 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:46.492005 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:46.492005 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:46.492005 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:46.492005 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:46.492005 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:46.492005 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:46.492005 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:46.492005 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:46.494068 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:46.494068 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:46.494068 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 13:33:46.494068 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 13:33:46.494068 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 13:33:46.494068 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 13:33:46.494068 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 13:33:46.494068 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 13:33:46.494068 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 26 13:33:46.494068 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:33:46.496822 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 345/681] 10.222.0.58 () {56 vars in 1096 bytes} [Tue Jul 26 13:33:46 2022] GET /identity/v3/users/swift => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:46.497832 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fd04dd-9184-4b83-bdef-1342d2fda64b None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:46.505531 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:46.551394 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c4024d-16df-4ebb-98c6-170b851dc5a6 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:46.596757 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524093f7-b3be-470c-bdec-6284a4f04a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:46.597388 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524093f7-b3be-470c-bdec-6284a4f04a94 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:46.597708 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524093f7-b3be-470c-bdec-6284a4f04a94 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:46.597991 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524093f7-b3be-470c-bdec-6284a4f04a94 None admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:46.598746 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524093f7-b3be-470c-bdec-6284a4f04a94 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:46.599059 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524093f7-b3be-470c-bdec-6284a4f04a94 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:46.600503 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524093f7-b3be-470c-bdec-6284a4f04a94 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:46.616922 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 337/682] 10.222.0.58 () {56 vars in 1148 bytes} [Tue Jul 26 13:33:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:46.617867 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524093f7-b3be-470c-bdec-6284a4f04a94 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:46.642408 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fd04dd-9184-4b83-bdef-1342d2fda64b None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:46.687270 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532f5441-3139-4310-94a8-448610e3d645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:46.688388 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532f5441-3139-4310-94a8-448610e3d645 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:46.688601 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532f5441-3139-4310-94a8-448610e3d645 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:46.688877 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532f5441-3139-4310-94a8-448610e3d645 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:46.697429 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532f5441-3139-4310-94a8-448610e3d645 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:46.699013 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532f5441-3139-4310-94a8-448610e3d645 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:46.713391 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-532f5441-3139-4310-94a8-448610e3d645 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 13:33:46.713391 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:46.713391 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:46.713391 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:46.713391 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:46.713391 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:46.713391 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:46.713391 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:46.713391 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:46.713391 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:46.713391 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:46.713391 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:46.713391 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 13:33:46.713391 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 13:33:46.713391 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 13:33:46.714382 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 13:33:46.714382 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:46.714382 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:46.714382 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:46.714382 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:46.714382 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:46.714382 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:46.714382 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:46.714382 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:46.714382 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:46.714382 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:46.714382 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:46.714382 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:46.714382 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:46.714382 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:46.714382 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:46.714382 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:46.720425 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 13:33:46.720425 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 13:33:46.720425 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 13:33:46.720425 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 13:33:46.720425 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 13:33:46.720425 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 13:33:46.720425 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 13:33:46.720425 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:33:46.720425 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 346/683] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Jul 26 13:33:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:46.720425 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532f5441-3139-4310-94a8-448610e3d645 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:46.749595 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524093f7-b3be-470c-bdec-6284a4f04a94 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:46.751443 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:46.793004 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175c42d6-eec4-42b8-b7bd-6cb2b59ef48e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:46.794182 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175c42d6-eec4-42b8-b7bd-6cb2b59ef48e None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:46.794595 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175c42d6-eec4-42b8-b7bd-6cb2b59ef48e None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:46.795152 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175c42d6-eec4-42b8-b7bd-6cb2b59ef48e None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:46.796329 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175c42d6-eec4-42b8-b7bd-6cb2b59ef48e None admin] RBAC: Adding query filter params (name=admin) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:46.797052 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175c42d6-eec4-42b8-b7bd-6cb2b59ef48e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:46.799339 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175c42d6-eec4-42b8-b7bd-6cb2b59ef48e None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:46.799895 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:46.817277 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 338/684] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Jul 26 13:33:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:46.818168 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175c42d6-eec4-42b8-b7bd-6cb2b59ef48e None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:46.855719 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532f5441-3139-4310-94a8-448610e3d645 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:46.895655 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7a502c-24e0-4424-a6e4-166ce3f9760e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:46.896285 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7a502c-24e0-4424-a6e4-166ce3f9760e None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:46.896570 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7a502c-24e0-4424-a6e4-166ce3f9760e None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:46.896844 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7a502c-24e0-4424-a6e4-166ce3f9760e None admin] PATH_INFO: `/v3/users` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:46.897693 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7a502c-24e0-4424-a6e4-166ce3f9760e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:46.897990 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7a502c-24e0-4424-a6e4-166ce3f9760e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:46.899466 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7a502c-24e0-4424-a6e4-166ce3f9760e None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:46.923633 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 347/685] 10.222.0.58 () {56 vars in 1135 bytes} [Tue Jul 26 13:33:46 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:46.924627 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7a502c-24e0-4424-a6e4-166ce3f9760e None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:46.945632 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175c42d6-eec4-42b8-b7bd-6cb2b59ef48e None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:46.992164 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f243ad6c-2ae2-447e-8e91-4595024aee1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:46.992974 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f243ad6c-2ae2-447e-8e91-4595024aee1f None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:46.993930 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f243ad6c-2ae2-447e-8e91-4595024aee1f None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:46.994308 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f243ad6c-2ae2-447e-8e91-4595024aee1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:46.995202 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f243ad6c-2ae2-447e-8e91-4595024aee1f None admin] RBAC: Adding query filter params (role.id=17e0f3c73a8a4a54b3e7b78878dfbf09, scope.project.id=1e8bb4385c204210a1bc77b1f0d654f3, user.id=72e400f267fd488d8439426a81ba0a25) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:46.995629 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f243ad6c-2ae2-447e-8e91-4595024aee1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:46.997162 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f243ad6c-2ae2-447e-8e91-4595024aee1f None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:47.003577 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 339/686] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Jul 26 13:33:46 2022] GET /identity/v3/role_assignments?user.id=72e400f267fd488d8439426a81ba0a25&scope.project.id=1e8bb4385c204210a1bc77b1f0d654f3&role.id=17e0f3c73a8a4a54b3e7b78878dfbf09 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:47.004532 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f243ad6c-2ae2-447e-8e91-4595024aee1f None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:47.066201 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:47.074898 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7a502c-24e0-4424-a6e4-166ce3f9760e None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:47.101400 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:47.120587 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec76ece5-5588-4c29-95a3-8574bc14c42e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:47.121242 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec76ece5-5588-4c29-95a3-8574bc14c42e None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:47.121581 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec76ece5-5588-4c29-95a3-8574bc14c42e None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:47.121827 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec76ece5-5588-4c29-95a3-8574bc14c42e None admin] PATH_INFO: `/v3/projects` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:47.123299 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec76ece5-5588-4c29-95a3-8574bc14c42e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:47.123621 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec76ece5-5588-4c29-95a3-8574bc14c42e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:47.125516 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec76ece5-5588-4c29-95a3-8574bc14c42e None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:47.144376 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 348/687] 10.222.0.58 () {56 vars in 1148 bytes} [Tue Jul 26 13:33:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:47.145460 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec76ece5-5588-4c29-95a3-8574bc14c42e None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:47.185451 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f243ad6c-2ae2-447e-8e91-4595024aee1f None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:47.226141 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68330b5d-96e7-4e84-ad21-985f60c05790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:47.227366 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68330b5d-96e7-4e84-ad21-985f60c05790 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:47.228195 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68330b5d-96e7-4e84-ad21-985f60c05790 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:47.228473 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68330b5d-96e7-4e84-ad21-985f60c05790 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:47.240516 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68330b5d-96e7-4e84-ad21-985f60c05790 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:47.242021 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68330b5d-96e7-4e84-ad21-985f60c05790 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:47.258915 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-68330b5d-96e7-4e84-ad21-985f60c05790 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:47.258915 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:47.258915 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:47.258915 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:47.258915 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:47.258915 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:47.258915 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:47.258915 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:47.258915 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:47.258915 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:47.258915 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:47.258915 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:47.258915 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 13:33:47.258915 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 13:33:47.258915 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 13:33:47.259881 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 13:33:47.259881 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:47.259881 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:47.259881 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:47.259881 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:47.259881 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:47.259881 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:47.259881 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:47.259881 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:47.259881 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:47.259881 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:47.259881 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:47.259881 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:47.259881 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:47.259881 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:47.259881 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:47.259881 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:47.260772 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 13:33:47.260772 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 13:33:47.260772 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:47.260772 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:33:47.260772 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 340/688] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:33:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:47.262019 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68330b5d-96e7-4e84-ad21-985f60c05790 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:47.295759 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec76ece5-5588-4c29-95a3-8574bc14c42e None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:47.353145 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55da06b1-ddc2-4683-99e4-f7e53d459352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:47.354129 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55da06b1-ddc2-4683-99e4-f7e53d459352 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:47.354583 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55da06b1-ddc2-4683-99e4-f7e53d459352 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:47.355483 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55da06b1-ddc2-4683-99e4-f7e53d459352 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:47.373748 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:47.398686 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68330b5d-96e7-4e84-ad21-985f60c05790 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:47.421982 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:47.446489 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e65631d-75c3-458f-b539-ed2804402273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:47.447125 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e65631d-75c3-458f-b539-ed2804402273 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:47.447819 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e65631d-75c3-458f-b539-ed2804402273 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:47.447819 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e65631d-75c3-458f-b539-ed2804402273 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:47.448445 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e65631d-75c3-458f-b539-ed2804402273 None admin] RBAC: Adding query filter params (role.id=17e0f3c73a8a4a54b3e7b78878dfbf09, scope.project.id=1e8bb4385c204210a1bc77b1f0d654f3, user.id=3223a78866d94c2ba66402911c2dfae9) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:47.448809 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e65631d-75c3-458f-b539-ed2804402273 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:47.450966 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55da06b1-ddc2-4683-99e4-f7e53d459352 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:47.451452 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e65631d-75c3-458f-b539-ed2804402273 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:47.452898 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55da06b1-ddc2-4683-99e4-f7e53d459352 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:47.465527 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 341/689] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Jul 26 13:33:47 2022] GET /identity/v3/role_assignments?user.id=3223a78866d94c2ba66402911c2dfae9&scope.project.id=1e8bb4385c204210a1bc77b1f0d654f3&role.id=17e0f3c73a8a4a54b3e7b78878dfbf09 => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:47.465527 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e65631d-75c3-458f-b539-ed2804402273 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:47.671241 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e65631d-75c3-458f-b539-ed2804402273 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:47.699486 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 349/690] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Jul 26 13:33:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 552 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 13:33:47.712552 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55da06b1-ddc2-4683-99e4-f7e53d459352 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:47.737751 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb437de-a41f-4b0f-a0c0-feb577609427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:47.739118 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb437de-a41f-4b0f-a0c0-feb577609427 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:47.739888 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb437de-a41f-4b0f-a0c0-feb577609427 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:47.740405 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb437de-a41f-4b0f-a0c0-feb577609427 None admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:47.741581 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb437de-a41f-4b0f-a0c0-feb577609427 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:47.742420 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb437de-a41f-4b0f-a0c0-feb577609427 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:47.744069 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb437de-a41f-4b0f-a0c0-feb577609427 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:47.765260 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 342/691] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Jul 26 13:33:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:47.771116 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb437de-a41f-4b0f-a0c0-feb577609427 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:47.779881 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:47.854759 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:47.894618 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55da06b1-ddc2-4683-99e4-f7e53d459352 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:47.941158 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb437de-a41f-4b0f-a0c0-feb577609427 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:47.985827 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1147ff08-de28-48ba-a3bd-3678a85d6856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:47.986848 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1147ff08-de28-48ba-a3bd-3678a85d6856 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:47.987136 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1147ff08-de28-48ba-a3bd-3678a85d6856 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:47.987558 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1147ff08-de28-48ba-a3bd-3678a85d6856 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:47.996556 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1147ff08-de28-48ba-a3bd-3678a85d6856 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:47.998450 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1147ff08-de28-48ba-a3bd-3678a85d6856 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:48.012878 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-1147ff08-de28-48ba-a3bd-3678a85d6856 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 26 13:33:48.012878 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:48.012878 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:48.012878 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:48.012878 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:48.012878 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:48.012878 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:48.012878 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:48.012878 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:48.012878 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:48.012878 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:48.012878 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:48.012878 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 13:33:48.012878 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 13:33:48.012878 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 13:33:48.012878 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 13:33:48.014012 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:48.014012 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:48.014012 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 13:33:48.014012 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 13:33:48.014012 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 13:33:48.014012 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 13:33:48.014012 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:48.014012 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:48.014012 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:48.014012 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:48.014012 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:48.014012 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:48.014012 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:48.014012 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:48.014012 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:48.014012 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:48.014012 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:48.014012 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:48.015476 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:48.015476 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:48.015476 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 13:33:48.015476 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 13:33:48.015476 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 13:33:48.015476 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 13:33:48.015476 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 13:33:48.015476 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 13:33:48.015476 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 26 13:33:48.015476 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:33:48.016027 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abe62b2-96a4-4ea6-8099-69fa49d6fa31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:48.016115 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 343/692] 10.222.0.58 () {56 vars in 1093 bytes} [Tue Jul 26 13:33:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 13:33:48.016183 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abe62b2-96a4-4ea6-8099-69fa49d6fa31 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:48.016380 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abe62b2-96a4-4ea6-8099-69fa49d6fa31 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:48.016608 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abe62b2-96a4-4ea6-8099-69fa49d6fa31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:48.021400 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1147ff08-de28-48ba-a3bd-3678a85d6856 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:48.054411 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abe62b2-96a4-4ea6-8099-69fa49d6fa31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:48.056422 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abe62b2-96a4-4ea6-8099-69fa49d6fa31 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:48.091754 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-1abe62b2-96a4-4ea6-8099-69fa49d6fa31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:48.091754 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:48.091754 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:48.091754 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:48.091754 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:48.091754 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:48.091754 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:48.091754 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:48.091754 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:48.091754 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:48.091754 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:48.091754 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:48.091754 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 13:33:48.091754 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 13:33:48.091754 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 13:33:48.092736 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 13:33:48.092736 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:48.092736 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:48.092736 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:48.092736 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:48.092736 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:48.092736 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:48.092736 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:48.092736 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:48.092736 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:48.092736 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:48.092736 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:48.092736 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:48.092736 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:48.092736 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:48.092736 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:48.092736 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:48.093685 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 13:33:48.093685 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 13:33:48.093685 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:48.093685 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:33:48.096093 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 350/693] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:33:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 385 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 13:33:48.103149 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abe62b2-96a4-4ea6-8099-69fa49d6fa31 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:48.145942 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:48.224353 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1147ff08-de28-48ba-a3bd-3678a85d6856 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:48.273503 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abe62b2-96a4-4ea6-8099-69fa49d6fa31 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:48.285211 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219cf745-fd0e-4308-820d-2be078e9f4ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:48.286217 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219cf745-fd0e-4308-820d-2be078e9f4ce None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:48.286700 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219cf745-fd0e-4308-820d-2be078e9f4ce None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:48.287196 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219cf745-fd0e-4308-820d-2be078e9f4ce None admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:48.288205 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219cf745-fd0e-4308-820d-2be078e9f4ce None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:48.288717 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219cf745-fd0e-4308-820d-2be078e9f4ce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:48.290919 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219cf745-fd0e-4308-820d-2be078e9f4ce None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:48.304664 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:48.317500 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20416c39-e82b-41b6-8c78-fe706605f781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:48.317500 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20416c39-e82b-41b6-8c78-fe706605f781 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:48.317500 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20416c39-e82b-41b6-8c78-fe706605f781 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:48.317500 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20416c39-e82b-41b6-8c78-fe706605f781 None admin] PATH_INFO: `/v3/domains` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:48.317921 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20416c39-e82b-41b6-8c78-fe706605f781 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:48.317921 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20416c39-e82b-41b6-8c78-fe706605f781 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:48.317921 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20416c39-e82b-41b6-8c78-fe706605f781 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:48.320707 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 344/694] 10.222.0.58 () {56 vars in 1133 bytes} [Tue Jul 26 13:33:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:48.327089 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219cf745-fd0e-4308-820d-2be078e9f4ce None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:48.330392 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 351/695] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Jul 26 13:33:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:48.336329 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20416c39-e82b-41b6-8c78-fe706605f781 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:48.454263 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219cf745-fd0e-4308-820d-2be078e9f4ce None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:48.482230 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:48.508674 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55e6c91-1541-4fec-8c22-56d55b01d27d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:48.509311 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55e6c91-1541-4fec-8c22-56d55b01d27d None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:48.509625 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55e6c91-1541-4fec-8c22-56d55b01d27d None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:48.509905 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55e6c91-1541-4fec-8c22-56d55b01d27d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:48.584869 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20416c39-e82b-41b6-8c78-fe706605f781 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:48.625248 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f662bef9-0176-49bc-aa32-eb1624ebcc01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:48.626123 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f662bef9-0176-49bc-aa32-eb1624ebcc01 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:48.626412 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f662bef9-0176-49bc-aa32-eb1624ebcc01 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:48.626690 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f662bef9-0176-49bc-aa32-eb1624ebcc01 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:48.634972 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f662bef9-0176-49bc-aa32-eb1624ebcc01 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:48.636492 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f662bef9-0176-49bc-aa32-eb1624ebcc01 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:48.650170 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-f662bef9-0176-49bc-aa32-eb1624ebcc01 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 13:33:48.650170 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:48.650170 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:48.650170 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:48.650170 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:48.650170 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:48.650170 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:48.650170 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:48.650170 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:48.650170 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:48.650170 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:48.650170 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:48.650170 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 13:33:48.650170 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 13:33:48.650170 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 13:33:48.651154 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 13:33:48.651154 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:48.651154 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:48.651154 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:48.651154 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:48.651154 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:48.651154 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:48.651154 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:48.651154 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:48.651154 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:48.651154 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:48.651154 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:48.651154 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:48.651154 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:48.651154 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:48.651154 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:48.651154 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:48.652024 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 13:33:48.652024 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 13:33:48.652024 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 13:33:48.652024 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 13:33:48.652024 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 13:33:48.652024 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 13:33:48.652024 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 13:33:48.652024 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:33:48.652700 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 352/696] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Jul 26 13:33:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 13:33:48.658942 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f662bef9-0176-49bc-aa32-eb1624ebcc01 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:48.725407 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55e6c91-1541-4fec-8c22-56d55b01d27d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:48.726903 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55e6c91-1541-4fec-8c22-56d55b01d27d None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:48.741947 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:48.809216 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f662bef9-0176-49bc-aa32-eb1624ebcc01 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:48.831452 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 345/697] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Jul 26 13:33:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 26 13:33:48.833077 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123bd054-df8d-48ed-a58e-a86a8ea6b01d None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:48.833466 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123bd054-df8d-48ed-a58e-a86a8ea6b01d None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:48.833869 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123bd054-df8d-48ed-a58e-a86a8ea6b01d None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:48.835587 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 346/698] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:33:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:33:48.841047 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123bd054-df8d-48ed-a58e-a86a8ea6b01d None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:48.851963 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f17023-7684-435c-9a80-4d53ee356e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:48.853165 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f17023-7684-435c-9a80-4d53ee356e5e None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:48.853448 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f17023-7684-435c-9a80-4d53ee356e5e None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:48.853756 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f17023-7684-435c-9a80-4d53ee356e5e None admin] PATH_INFO: `/v3/projects` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:48.855237 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f17023-7684-435c-9a80-4d53ee356e5e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:48.856225 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f17023-7684-435c-9a80-4d53ee356e5e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:48.860155 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f17023-7684-435c-9a80-4d53ee356e5e None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:48.869920 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:48.877023 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 353/699] 10.222.0.58 () {56 vars in 1148 bytes} [Tue Jul 26 13:33:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:48.878672 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9fb677-2f38-423b-ace5-c17a5a286312 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:48.878969 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9fb677-2f38-423b-ace5-c17a5a286312 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:48.879478 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9fb677-2f38-423b-ace5-c17a5a286312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:48.901118 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-6c9fb677-2f38-423b-ace5-c17a5a286312 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:33:48.903032 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-6c9fb677-2f38-423b-ace5-c17a5a286312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:33:48.972757 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123bd054-df8d-48ed-a58e-a86a8ea6b01d None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:48.997148 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 354/700] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:33:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 13:33:48.997987 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9fb677-2f38-423b-ace5-c17a5a286312 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:49.023329 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa10a3cb-4960-4cc4-9f5e-7b5b205eeb15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:49.024916 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa10a3cb-4960-4cc4-9f5e-7b5b205eeb15 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:49.025422 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa10a3cb-4960-4cc4-9f5e-7b5b205eeb15 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:49.025942 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa10a3cb-4960-4cc4-9f5e-7b5b205eeb15 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:49.042488 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa10a3cb-4960-4cc4-9f5e-7b5b205eeb15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:49.045193 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa10a3cb-4960-4cc4-9f5e-7b5b205eeb15 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:49.050558 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:49.070097 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-fa10a3cb-4960-4cc4-9f5e-7b5b205eeb15 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:49.070097 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:49.070097 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:49.070097 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:49.070097 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:49.070097 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:49.070097 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:49.070097 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:49.070097 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:49.070097 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:49.070097 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:49.070097 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:49.070097 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 13:33:49.070097 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 13:33:49.070097 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 13:33:49.071730 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 13:33:49.071730 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:49.071730 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:49.071730 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:49.071730 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:49.071730 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:49.071730 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:49.071730 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:49.071730 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:49.071730 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:49.071730 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:49.071730 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:49.071730 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:49.071730 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:49.071730 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:49.071730 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:49.071730 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:49.072642 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 13:33:49.072642 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 13:33:49.072642 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:49.072642 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:33:49.072880 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 347/701] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:33:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:49.077258 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa10a3cb-4960-4cc4-9f5e-7b5b205eeb15 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:49.124486 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9fb677-2f38-423b-ace5-c17a5a286312 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:49.169764 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f5710a-315a-4173-8e65-8cfc678f0a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:49.170393 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f5710a-315a-4173-8e65-8cfc678f0a49 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:49.170830 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f5710a-315a-4173-8e65-8cfc678f0a49 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:49.171106 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f5710a-315a-4173-8e65-8cfc678f0a49 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:49.173046 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f5710a-315a-4173-8e65-8cfc678f0a49 None admin] RBAC: Adding query filter params (role.id=17e0f3c73a8a4a54b3e7b78878dfbf09, scope.project.id=1e8bb4385c204210a1bc77b1f0d654f3, user.id=3ab2819bac844163ae4305e4f2186605) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:49.173363 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f5710a-315a-4173-8e65-8cfc678f0a49 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:49.174832 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f5710a-315a-4173-8e65-8cfc678f0a49 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:49.177960 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:49.189503 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 355/702] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Jul 26 13:33:48 2022] GET /identity/v3/role_assignments?user.id=3ab2819bac844163ae4305e4f2186605&scope.project.id=1e8bb4385c204210a1bc77b1f0d654f3&role.id=17e0f3c73a8a4a54b3e7b78878dfbf09 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:49.189503 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f5710a-315a-4173-8e65-8cfc678f0a49 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:49.238773 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa10a3cb-4960-4cc4-9f5e-7b5b205eeb15 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:49.334042 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f5710a-315a-4173-8e65-8cfc678f0a49 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:49.337994 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f50b975-d17e-4b7d-b26b-4a0f4950b954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:49.338965 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f50b975-d17e-4b7d-b26b-4a0f4950b954 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:49.339450 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f50b975-d17e-4b7d-b26b-4a0f4950b954 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:49.339924 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f50b975-d17e-4b7d-b26b-4a0f4950b954 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:49.342742 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:49.365983 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f50b975-d17e-4b7d-b26b-4a0f4950b954 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:49.367656 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f50b975-d17e-4b7d-b26b-4a0f4950b954 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:49.376832 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea09ff8a-565f-4280-8e21-bccd564c83cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:49.377765 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea09ff8a-565f-4280-8e21-bccd564c83cc None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:49.378218 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea09ff8a-565f-4280-8e21-bccd564c83cc None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:49.378637 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea09ff8a-565f-4280-8e21-bccd564c83cc None admin] PATH_INFO: `/v3/domains` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:49.379539 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea09ff8a-565f-4280-8e21-bccd564c83cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:49.380554 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea09ff8a-565f-4280-8e21-bccd564c83cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:49.382167 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea09ff8a-565f-4280-8e21-bccd564c83cc None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:49.398192 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 356/703] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Jul 26 13:33:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:49.403630 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-1f50b975-d17e-4b7d-b26b-4a0f4950b954 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:49.403630 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:49.403630 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:49.403630 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:49.403630 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:49.403630 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:49.403630 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:49.403630 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:49.403630 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:49.403630 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:49.403630 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:49.403630 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:49.403630 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 13:33:49.403630 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 13:33:49.403630 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 13:33:49.404769 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 13:33:49.404769 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:49.404769 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:49.404769 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:49.404769 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:49.404769 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:49.404769 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:49.404769 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:49.404769 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:49.404769 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:49.404769 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:49.404769 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:49.404769 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:49.404769 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:49.404769 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:49.404769 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:49.404769 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:49.405956 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 13:33:49.405956 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 13:33:49.405956 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:49.405956 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:33:49.407230 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea09ff8a-565f-4280-8e21-bccd564c83cc None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:49.411561 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 348/704] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:33:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:49.415258 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f50b975-d17e-4b7d-b26b-4a0f4950b954 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:49.558486 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f50b975-d17e-4b7d-b26b-4a0f4950b954 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:49.559794 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea09ff8a-565f-4280-8e21-bccd564c83cc None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:49.602713 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:49.609660 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00b5052-0147-4c76-b60a-561beb6e040d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:49.610778 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00b5052-0147-4c76-b60a-561beb6e040d None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:49.611270 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00b5052-0147-4c76-b60a-561beb6e040d None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:49.611854 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00b5052-0147-4c76-b60a-561beb6e040d None admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:49.612770 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00b5052-0147-4c76-b60a-561beb6e040d None admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:49.613310 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00b5052-0147-4c76-b60a-561beb6e040d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:49.614803 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00b5052-0147-4c76-b60a-561beb6e040d None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:49.618385 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297f16d6-3ad0-48e8-a700-17ebf110dac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:49.619052 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297f16d6-3ad0-48e8-a700-17ebf110dac2 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:49.619329 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297f16d6-3ad0-48e8-a700-17ebf110dac2 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:49.619593 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297f16d6-3ad0-48e8-a700-17ebf110dac2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:49.628212 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297f16d6-3ad0-48e8-a700-17ebf110dac2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:49.629721 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 349/705] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Jul 26 13:33:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:49.629836 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297f16d6-3ad0-48e8-a700-17ebf110dac2 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:49.630852 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb417008-181b-47e7-903e-6b68a83a9d2e None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:49.631143 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb417008-181b-47e7-903e-6b68a83a9d2e None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:49.631413 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb417008-181b-47e7-903e-6b68a83a9d2e None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:49.633450 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 350/706] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:33:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:33:49.636044 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb417008-181b-47e7-903e-6b68a83a9d2e None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:49.644890 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-297f16d6-3ad0-48e8-a700-17ebf110dac2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 13:33:49.644890 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:49.644890 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:49.644890 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:49.644890 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:49.644890 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:49.644890 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:49.644890 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:49.644890 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:49.644890 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:49.644890 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:49.644890 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:49.644890 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 13:33:49.644890 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 13:33:49.644890 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 13:33:49.646110 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 13:33:49.646110 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:49.646110 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:49.646110 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:49.646110 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:49.646110 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:49.646110 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:49.646110 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:49.646110 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:49.646110 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:49.646110 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:49.646110 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:49.646110 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:49.646110 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:49.646110 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:49.646110 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:49.646110 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:49.647633 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 13:33:49.647633 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 13:33:49.647633 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 13:33:49.647633 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 13:33:49.647633 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 13:33:49.647633 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 13:33:49.647633 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 13:33:49.647633 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:33:49.647633 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 357/707] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Jul 26 13:33:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:49.647633 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d86814-df33-4f5c-85ca-6bee99460891 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:49.648686 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d86814-df33-4f5c-85ca-6bee99460891 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:49.648686 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d86814-df33-4f5c-85ca-6bee99460891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:49.678693 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-a5d86814-df33-4f5c-85ca-6bee99460891 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:33:49.682118 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-a5d86814-df33-4f5c-85ca-6bee99460891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:33:49.714909 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:49.770547 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb417008-181b-47e7-903e-6b68a83a9d2e None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:49.787132 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 358/708] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:33:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 13:33:49.788961 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d86814-df33-4f5c-85ca-6bee99460891 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:49.810905 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4055554-9d63-4774-ae9c-51f96402a700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:49.812011 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4055554-9d63-4774-ae9c-51f96402a700 None admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:49.812541 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4055554-9d63-4774-ae9c-51f96402a700 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:49.812990 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4055554-9d63-4774-ae9c-51f96402a700 None admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:49.813883 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4055554-9d63-4774-ae9c-51f96402a700 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:49.815616 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4055554-9d63-4774-ae9c-51f96402a700 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:49.840702 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 351/709] 10.222.0.58 () {60 vars in 1132 bytes} [Tue Jul 26 13:33:49 2022] POST /identity/v3/users => generated 254 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:33:49.842016 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4055554-9d63-4774-ae9c-51f96402a700 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:49.895856 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:49.954512 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d86814-df33-4f5c-85ca-6bee99460891 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:49.967722 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4055554-9d63-4774-ae9c-51f96402a700 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:50.011268 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eba8a0-7c71-4c27-a6b1-6d3da59b9f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:50.012006 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eba8a0-7c71-4c27-a6b1-6d3da59b9f7b None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:50.012312 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eba8a0-7c71-4c27-a6b1-6d3da59b9f7b None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:50.012587 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eba8a0-7c71-4c27-a6b1-6d3da59b9f7b None admin] PATH_INFO: `/v3/projects` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:50.013522 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eba8a0-7c71-4c27-a6b1-6d3da59b9f7b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:50.013824 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eba8a0-7c71-4c27-a6b1-6d3da59b9f7b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:50.015741 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eba8a0-7c71-4c27-a6b1-6d3da59b9f7b None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:50.016404 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aa3c33-7e51-469d-b6a0-16238799a894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:50.017333 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aa3c33-7e51-469d-b6a0-16238799a894 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:50.017902 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aa3c33-7e51-469d-b6a0-16238799a894 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:50.018370 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aa3c33-7e51-469d-b6a0-16238799a894 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:50.019356 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8aa3c33-7e51-469d-b6a0-16238799a894 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:50.021151 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8aa3c33-7e51-469d-b6a0-16238799a894 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:50.029929 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 359/710] 10.222.0.58 () {56 vars in 1148 bytes} [Tue Jul 26 13:33:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:50.039280 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-f8aa3c33-7e51-469d-b6a0-16238799a894 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 26 13:33:50.039280 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:50.039280 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 13:33:50.039280 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 13:33:50.039280 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 13:33:50.039280 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 13:33:50.039280 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 13:33:50.039280 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 13:33:50.039280 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 26 13:33:50.039280 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:33:50.039280 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 13:33:50.039280 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:33:50.039280 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:50.039280 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:50.039280 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:50.039280 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:50.040889 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:50.040889 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:50.040889 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:50.040889 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:50.040889 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:50.040889 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:50.040889 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:50.040889 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 13:33:50.040889 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 13:33:50.040889 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 13:33:50.040889 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 13:33:50.040889 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:50.040889 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:50.040889 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:50.040889 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:50.040889 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:50.041786 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:50.041786 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:50.041786 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:50.041786 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:50.041786 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:50.041786 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:50.041786 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:50.041786 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:50.041786 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:50.041786 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:50.041786 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:50.041786 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 13:33:50.041786 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 13:33:50.041786 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 26 13:33:50.041786 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:33:50.041786 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 352/711] 10.222.0.58 () {56 vars in 1129 bytes} [Tue Jul 26 13:33:49 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:50.048239 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eba8a0-7c71-4c27-a6b1-6d3da59b9f7b None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:50.050511 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aa3c33-7e51-469d-b6a0-16238799a894 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:50.073008 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:50.191676 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eba8a0-7c71-4c27-a6b1-6d3da59b9f7b None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:50.196234 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aa3c33-7e51-469d-b6a0-16238799a894 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:50.206707 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:50.247605 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9f631f-70ed-4998-a1ce-2b04d77508ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:50.248247 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9f631f-70ed-4998-a1ce-2b04d77508ef None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:50.248525 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9f631f-70ed-4998-a1ce-2b04d77508ef None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:50.248796 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9f631f-70ed-4998-a1ce-2b04d77508ef None admin] PATH_INFO: `/v3/services` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:50.249611 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9f631f-70ed-4998-a1ce-2b04d77508ef None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:50.249914 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9f631f-70ed-4998-a1ce-2b04d77508ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:50.251311 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9f631f-70ed-4998-a1ce-2b04d77508ef None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:50.256054 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b15fedb-3d3d-47aa-a643-041dc689a7c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:50.259924 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 360/712] 10.222.0.58 () {56 vars in 1124 bytes} [Tue Jul 26 13:33:50 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:50.260073 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b15fedb-3d3d-47aa-a643-041dc689a7c2 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:50.260073 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b15fedb-3d3d-47aa-a643-041dc689a7c2 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:50.260073 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b15fedb-3d3d-47aa-a643-041dc689a7c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:50.260263 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49390bf2-c928-452e-830b-8fa87c341d16 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:50.260353 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b15fedb-3d3d-47aa-a643-041dc689a7c2 None admin] RBAC: Adding query filter params (role.id=6e0201a9a6484f7d8b112b1eafff70bd, scope.project.id=1e8bb4385c204210a1bc77b1f0d654f3, user.id=c69161cd520b4b29bf50f3bf2f1c033b) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:50.260427 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49390bf2-c928-452e-830b-8fa87c341d16 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:50.260503 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b15fedb-3d3d-47aa-a643-041dc689a7c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:50.260572 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49390bf2-c928-452e-830b-8fa87c341d16 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:50.260640 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b15fedb-3d3d-47aa-a643-041dc689a7c2 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:50.260706 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 361/713] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:33:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:33:50.266947 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 353/714] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Jul 26 13:33:50 2022] GET /identity/v3/role_assignments?user.id=c69161cd520b4b29bf50f3bf2f1c033b&scope.project.id=1e8bb4385c204210a1bc77b1f0d654f3&role.id=6e0201a9a6484f7d8b112b1eafff70bd => generated 259 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:50.267895 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f289c9b-2952-4633-a3ce-4fcb91f85b9d None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:50.268184 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f289c9b-2952-4633-a3ce-4fcb91f85b9d None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:50.268462 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f289c9b-2952-4633-a3ce-4fcb91f85b9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:50.268912 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b15fedb-3d3d-47aa-a643-041dc689a7c2 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:50.308762 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-1f289c9b-2952-4633-a3ce-4fcb91f85b9d None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:33:50.310090 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-1f289c9b-2952-4633-a3ce-4fcb91f85b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:33:50.465277 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 362/715] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:33:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 13:33:50.484841 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b15fedb-3d3d-47aa-a643-041dc689a7c2 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:50.490230 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f289c9b-2952-4633-a3ce-4fcb91f85b9d None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:50.506346 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:50.559925 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84934784-ab48-4eda-a367-9c1135016e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:50.560945 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84934784-ab48-4eda-a367-9c1135016e4d None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:50.562988 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84934784-ab48-4eda-a367-9c1135016e4d None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:50.563498 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84934784-ab48-4eda-a367-9c1135016e4d None admin] PATH_INFO: `/v3/services` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:50.564441 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84934784-ab48-4eda-a367-9c1135016e4d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:50.565797 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84934784-ab48-4eda-a367-9c1135016e4d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:50.568886 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84934784-ab48-4eda-a367-9c1135016e4d None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:50.592775 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 354/716] 10.222.0.58 () {56 vars in 1124 bytes} [Tue Jul 26 13:33:50 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:50.703235 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f289c9b-2952-4633-a3ce-4fcb91f85b9d None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:50.768458 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c6688a-3bd4-4a8a-b6f3-cacb4f5215f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:50.769049 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c6688a-3bd4-4a8a-b6f3-cacb4f5215f4 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:50.769331 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c6688a-3bd4-4a8a-b6f3-cacb4f5215f4 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:50.769801 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c6688a-3bd4-4a8a-b6f3-cacb4f5215f4 None admin] PATH_INFO: `/v3/services/network` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:50.770446 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c6688a-3bd4-4a8a-b6f3-cacb4f5215f4 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:50.771825 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c6688a-3bd4-4a8a-b6f3-cacb4f5215f4 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:50.790025 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-34c6688a-3bd4-4a8a-b6f3-cacb4f5215f4 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 26 13:33:50.790025 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:50.790025 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 13:33:50.790025 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 13:33:50.790025 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 13:33:50.790025 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 13:33:50.790025 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 13:33:50.790025 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 13:33:50.790025 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 26 13:33:50.790025 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:33:50.790025 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 13:33:50.790025 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:33:50.790025 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:50.790025 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:50.790025 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:50.790025 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:50.790025 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:50.791513 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:50.791513 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:50.791513 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:50.791513 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:50.791513 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:50.791513 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:50.791513 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 13:33:50.791513 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 13:33:50.791513 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 13:33:50.791513 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 13:33:50.791513 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:50.791513 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:50.791513 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:50.791513 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:50.791513 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:50.791513 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:50.792350 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:50.792350 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:50.792350 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:50.792350 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:50.792350 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:50.792350 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:50.792350 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:50.792350 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:50.792350 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:50.792350 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:50.792350 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 13:33:50.792350 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 13:33:50.792350 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 26 13:33:50.792350 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:33:50.792350 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 363/717] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Jul 26 13:33:50 2022] GET /identity/v3/services/network => generated 88 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 13:33:50.799306 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84934784-ab48-4eda-a367-9c1135016e4d None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:50.852797 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:50.946985 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84934784-ab48-4eda-a367-9c1135016e4d None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:50.995861 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dcbdbf-9abb-4ea8-9aa9-16ac78ae7696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:50.996887 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dcbdbf-9abb-4ea8-9aa9-16ac78ae7696 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:50.997374 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dcbdbf-9abb-4ea8-9aa9-16ac78ae7696 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:50.997884 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dcbdbf-9abb-4ea8-9aa9-16ac78ae7696 None admin] PATH_INFO: `/v3/services` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:50.998771 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dcbdbf-9abb-4ea8-9aa9-16ac78ae7696 None admin] RBAC: Adding query filter params (name=network) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:50.999277 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dcbdbf-9abb-4ea8-9aa9-16ac78ae7696 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:51.000858 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dcbdbf-9abb-4ea8-9aa9-16ac78ae7696 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:51.009961 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 355/718] 10.222.0.58 () {56 vars in 1112 bytes} [Tue Jul 26 13:33:50 2022] GET /identity/v3/services?name=network => generated 124 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:51.015553 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c6688a-3bd4-4a8a-b6f3-cacb4f5215f4 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:51.137432 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:51.179015 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c6688a-3bd4-4a8a-b6f3-cacb4f5215f4 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:51.233652 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa7f3eb-db05-4c2f-842a-51effcc332e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:51.234609 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa7f3eb-db05-4c2f-842a-51effcc332e5 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:51.235064 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa7f3eb-db05-4c2f-842a-51effcc332e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:51.235470 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa7f3eb-db05-4c2f-842a-51effcc332e5 None admin] PATH_INFO: `/v3/services` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:51.236607 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa7f3eb-db05-4c2f-842a-51effcc332e5 None admin] RBAC: Adding query filter params (type=network) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:51.236926 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa7f3eb-db05-4c2f-842a-51effcc332e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:51.238371 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa7f3eb-db05-4c2f-842a-51effcc332e5 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:51.245580 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 364/719] 10.222.0.58 () {56 vars in 1112 bytes} [Tue Jul 26 13:33:51 2022] GET /identity/v3/services?type=network => generated 124 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:51.707556 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcdb96e-251b-49d7-a06b-f23a38ba2f0a None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:51.708272 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcdb96e-251b-49d7-a06b-f23a38ba2f0a None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:51.708787 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcdb96e-251b-49d7-a06b-f23a38ba2f0a None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:51.710670 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 356/720] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:33:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:33:51.717174 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13845748-2166-455e-affd-b564a0a68d1c None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:51.717919 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13845748-2166-455e-affd-b564a0a68d1c None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:51.718479 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13845748-2166-455e-affd-b564a0a68d1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:51.739947 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-13845748-2166-455e-affd-b564a0a68d1c None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:33:51.741658 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-13845748-2166-455e-affd-b564a0a68d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:33:51.940611 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 365/721] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:33:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 13:33:51.958634 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcdb96e-251b-49d7-a06b-f23a38ba2f0a None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:52.126820 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:52.134533 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcdb96e-251b-49d7-a06b-f23a38ba2f0a None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:52.188204 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3ab8b9-d89c-422c-bb86-2ffb8d442d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:52.189305 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3ab8b9-d89c-422c-bb86-2ffb8d442d29 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:52.189789 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3ab8b9-d89c-422c-bb86-2ffb8d442d29 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:52.190243 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3ab8b9-d89c-422c-bb86-2ffb8d442d29 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:52.206955 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3ab8b9-d89c-422c-bb86-2ffb8d442d29 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:33:52.213178 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-bb3ab8b9-d89c-422c-bb86-2ffb8d442d29 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 13:33:52.213178 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:52.213178 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:52.213178 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:52.213178 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:52.213178 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:52.213178 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:52.213178 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:52.213178 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:52.213178 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:52.213178 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:52.213178 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:52.213178 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:33:52.213178 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:33:52.213178 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:33:52.213178 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:33:52.214159 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:33:52.214159 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:33:52.214159 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:33:52.214159 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:33:52.214159 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:52.214159 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:52.214159 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:52.214159 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:52.214159 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:52.214159 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:52.214159 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:52.214159 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:52.214159 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:52.214159 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:52.214159 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:52.214159 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:52.215407 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:52.215407 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:52.215407 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:52.215407 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:52.215407 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:33:52.215407 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:33:52.215407 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:33:52.215407 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:33:52.215407 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:33:52.215407 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:33:52.215407 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 13:33:52.215407 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:33:52.215407 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 357/722] 10.222.0.58 () {56 vars in 1096 bytes} [Tue Jul 26 13:33:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:52.222343 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13845748-2166-455e-affd-b564a0a68d1c None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:52.321896 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:52.365610 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13845748-2166-455e-affd-b564a0a68d1c None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:52.406841 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265211c0-26b9-44c9-bf05-69e406d97035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:52.407477 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265211c0-26b9-44c9-bf05-69e406d97035 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:52.407762 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265211c0-26b9-44c9-bf05-69e406d97035 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:52.408038 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265211c0-26b9-44c9-bf05-69e406d97035 None admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:52.408772 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265211c0-26b9-44c9-bf05-69e406d97035 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:52.409082 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265211c0-26b9-44c9-bf05-69e406d97035 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:52.411257 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265211c0-26b9-44c9-bf05-69e406d97035 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:52.421853 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 366/723] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Jul 26 13:33:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:52.431301 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3ab8b9-d89c-422c-bb86-2ffb8d442d29 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:52.595624 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3ab8b9-d89c-422c-bb86-2ffb8d442d29 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:52.650207 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb44ee2-447a-407b-a7bc-839b95448b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:52.651258 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb44ee2-447a-407b-a7bc-839b95448b35 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:52.651747 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb44ee2-447a-407b-a7bc-839b95448b35 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:52.652260 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb44ee2-447a-407b-a7bc-839b95448b35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:52.669486 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb44ee2-447a-407b-a7bc-839b95448b35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:52.671307 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb44ee2-447a-407b-a7bc-839b95448b35 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:52.689618 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-8fb44ee2-447a-407b-a7bc-839b95448b35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:52.689618 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:52.689618 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:52.689618 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:52.689618 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:52.689618 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:52.689618 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:52.689618 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:52.689618 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:52.689618 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:52.689618 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:52.689618 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:52.689618 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 13:33:52.689618 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 13:33:52.689618 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 13:33:52.690573 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 13:33:52.690573 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:52.690573 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:52.690573 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:52.690573 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:52.690573 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:52.690573 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:52.690573 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:52.690573 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:52.690573 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:52.690573 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:52.690573 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:52.690573 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:52.690573 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:52.690573 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:52.690573 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:52.690573 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:52.691526 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 13:33:52.691526 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 13:33:52.691526 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:52.691526 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:33:52.693451 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 358/724] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:33:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:52.699888 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265211c0-26b9-44c9-bf05-69e406d97035 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:52.863349 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265211c0-26b9-44c9-bf05-69e406d97035 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:52.897293 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3f53e5-eee7-49f1-9d77-812af09be324 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:52.897664 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3f53e5-eee7-49f1-9d77-812af09be324 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:52.897880 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3f53e5-eee7-49f1-9d77-812af09be324 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:52.899740 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 359/725] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:33:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 13:33:52.904916 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05559f4a-fb44-4cb3-9e77-d419306d42e2 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:52.905784 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05559f4a-fb44-4cb3-9e77-d419306d42e2 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:52.905784 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05559f4a-fb44-4cb3-9e77-d419306d42e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:52.905981 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:52.912855 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f79ff2-9fd7-4be5-9cf6-ca222ec9d913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:52.913434 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f79ff2-9fd7-4be5-9cf6-ca222ec9d913 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:52.913753 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f79ff2-9fd7-4be5-9cf6-ca222ec9d913 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:52.914225 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f79ff2-9fd7-4be5-9cf6-ca222ec9d913 None admin] PATH_INFO: `/v3/domains` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:52.915484 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f79ff2-9fd7-4be5-9cf6-ca222ec9d913 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:52.915997 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f79ff2-9fd7-4be5-9cf6-ca222ec9d913 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:52.917378 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f79ff2-9fd7-4be5-9cf6-ca222ec9d913 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:52.930213 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 367/726] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Jul 26 13:33:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:52.936219 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f79ff2-9fd7-4be5-9cf6-ca222ec9d913 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:52.938616 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-05559f4a-fb44-4cb3-9e77-d419306d42e2 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:33:52.940101 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-05559f4a-fb44-4cb3-9e77-d419306d42e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:33:52.948591 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:53.050752 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 360/727] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:33:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 13:33:53.052769 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3daa487-0ef0-43d6-9ce3-2f482604fa60 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:53.053273 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3daa487-0ef0-43d6-9ce3-2f482604fa60 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:53.054062 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3daa487-0ef0-43d6-9ce3-2f482604fa60 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:53.057363 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 361/728] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:33:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:33:53.063031 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8cec04-0775-419b-a8d2-88c8baaaa924 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:53.063516 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8cec04-0775-419b-a8d2-88c8baaaa924 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:53.064014 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8cec04-0775-419b-a8d2-88c8baaaa924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:53.085645 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f79ff2-9fd7-4be5-9cf6-ca222ec9d913 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:53.090861 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-0d8cec04-0775-419b-a8d2-88c8baaaa924 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:33:53.092431 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-0d8cec04-0775-419b-a8d2-88c8baaaa924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:33:53.136148 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bb0e0c-e64f-40d3-a0a7-ae22050cd011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:53.136720 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bb0e0c-e64f-40d3-a0a7-ae22050cd011 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:53.137001 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bb0e0c-e64f-40d3-a0a7-ae22050cd011 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:53.137273 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bb0e0c-e64f-40d3-a0a7-ae22050cd011 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:53.149597 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bb0e0c-e64f-40d3-a0a7-ae22050cd011 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:53.151850 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bb0e0c-e64f-40d3-a0a7-ae22050cd011 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:53.168537 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-b1bb0e0c-e64f-40d3-a0a7-ae22050cd011 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 26 13:33:53.168537 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:53.168537 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:53.168537 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:53.168537 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:53.168537 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:53.168537 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:53.168537 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:53.168537 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:53.168537 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:53.168537 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:53.168537 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:53.168537 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 13:33:53.168537 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 13:33:53.168537 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 13:33:53.168537 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 13:33:53.170257 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:53.170257 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:53.170257 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 13:33:53.170257 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 13:33:53.170257 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 13:33:53.170257 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 13:33:53.170257 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:53.170257 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:53.170257 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:53.170257 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:53.170257 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:53.170257 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:53.170257 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:53.170257 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:53.170257 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:53.170257 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:53.170257 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:53.170257 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:53.172811 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:53.172811 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:53.172811 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 13:33:53.172811 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 13:33:53.172811 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 13:33:53.172811 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 13:33:53.172811 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 13:33:53.172811 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 13:33:53.172811 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 26 13:33:53.172811 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:33:53.172811 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 368/729] 10.222.0.58 () {56 vars in 1096 bytes} [Tue Jul 26 13:33:52 2022] GET /identity/v3/users/swift => generated 83 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:53.172811 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bb0e0c-e64f-40d3-a0a7-ae22050cd011 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:53.176555 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 362/730] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:33:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 13:33:53.177556 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8cec04-0775-419b-a8d2-88c8baaaa924 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:53.250217 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:53.266148 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:53.304224 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8cec04-0775-419b-a8d2-88c8baaaa924 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:53.305529 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bb0e0c-e64f-40d3-a0a7-ae22050cd011 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:53.340813 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8b2ba9-cbe0-4dd2-b5b6-69fb9dc85f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:53.341441 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8b2ba9-cbe0-4dd2-b5b6-69fb9dc85f26 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:53.341751 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8b2ba9-cbe0-4dd2-b5b6-69fb9dc85f26 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:53.342078 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8b2ba9-cbe0-4dd2-b5b6-69fb9dc85f26 None admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:53.342884 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8b2ba9-cbe0-4dd2-b5b6-69fb9dc85f26 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:53.343252 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8b2ba9-cbe0-4dd2-b5b6-69fb9dc85f26 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:53.344682 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8b2ba9-cbe0-4dd2-b5b6-69fb9dc85f26 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:53.349202 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ea7a97-1b99-4b2d-b45f-b8b7bd750f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:53.350122 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ea7a97-1b99-4b2d-b45f-b8b7bd750f12 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:53.350619 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ea7a97-1b99-4b2d-b45f-b8b7bd750f12 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:53.351093 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ea7a97-1b99-4b2d-b45f-b8b7bd750f12 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:53.368861 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 363/731] 10.222.0.58 () {56 vars in 1135 bytes} [Tue Jul 26 13:33:53 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:53.368861 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8b2ba9-cbe0-4dd2-b5b6-69fb9dc85f26 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:53.370049 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ea7a97-1b99-4b2d-b45f-b8b7bd750f12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:33:53.375696 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-83ea7a97-1b99-4b2d-b45f-b8b7bd750f12 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 13:33:53.375696 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:53.375696 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:53.375696 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:53.375696 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:53.375696 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:53.375696 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:53.375696 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:53.375696 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:53.375696 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:53.375696 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:53.375696 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:53.375696 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:33:53.375696 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:33:53.375696 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:33:53.375696 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:33:53.377347 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:33:53.377347 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:33:53.377347 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:33:53.377347 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:33:53.377347 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:53.377347 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:53.377347 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:53.377347 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:53.377347 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:53.377347 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:53.377347 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:53.377347 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:53.377347 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:53.377347 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:53.377347 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:53.377347 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:53.378935 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:53.378935 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:53.378935 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:53.378935 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:53.378935 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:33:53.378935 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:33:53.378935 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:33:53.378935 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:33:53.378935 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:33:53.378935 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:33:53.378935 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 13:33:53.378935 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:33:53.378935 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 369/732] 10.222.0.58 () {56 vars in 1096 bytes} [Tue Jul 26 13:33:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:53.378935 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb1aebe-3dff-4319-9443-0521103eddf6 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:53.379681 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb1aebe-3dff-4319-9443-0521103eddf6 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:53.379681 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb1aebe-3dff-4319-9443-0521103eddf6 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:53.384384 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 370/733] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:33:53 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:33:53.385037 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb1aebe-3dff-4319-9443-0521103eddf6 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:53.482900 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8b2ba9-cbe0-4dd2-b5b6-69fb9dc85f26 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:53.497666 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb1aebe-3dff-4319-9443-0521103eddf6 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:53.509508 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:53.517899 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:53.526978 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a3b4ad-ceda-45b2-b4af-f3b551ac239b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:53.527738 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a3b4ad-ceda-45b2-b4af-f3b551ac239b None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:53.528019 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a3b4ad-ceda-45b2-b4af-f3b551ac239b None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:53.528289 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a3b4ad-ceda-45b2-b4af-f3b551ac239b None admin] PATH_INFO: `/v3/roles/service` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:53.543172 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a3b4ad-ceda-45b2-b4af-f3b551ac239b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:33:53.544585 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0378473-b22f-4dde-8e77-5356a0d111d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:53.545213 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0378473-b22f-4dde-8e77-5356a0d111d2 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:53.545515 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0378473-b22f-4dde-8e77-5356a0d111d2 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:53.545793 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0378473-b22f-4dde-8e77-5356a0d111d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:53.548473 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-e4a3b4ad-ceda-45b2-b4af-f3b551ac239b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 13:33:53.548473 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:53.548473 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:53.548473 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:53.548473 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:53.548473 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:53.548473 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:53.548473 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:53.548473 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:53.548473 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:53.548473 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:53.548473 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:53.548473 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:33:53.548473 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:33:53.548473 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:33:53.548473 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:33:53.549360 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:33:53.549360 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:33:53.549360 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:33:53.549360 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:33:53.549360 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:53.549360 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:53.549360 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:53.549360 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:53.549360 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:53.549360 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:53.549360 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:53.549360 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:53.549360 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:53.549360 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:53.549360 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:53.549360 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:53.550221 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:53.550221 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:53.550221 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:53.550221 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:53.550221 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:33:53.550221 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:33:53.550221 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:33:53.550221 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:33:53.550221 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:33:53.550221 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:33:53.550221 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 13:33:53.550221 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:33:53.550221 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 364/734] 10.222.0.58 () {56 vars in 1102 bytes} [Tue Jul 26 13:33:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:53.550929 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a3b4ad-ceda-45b2-b4af-f3b551ac239b None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:53.639588 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0378473-b22f-4dde-8e77-5356a0d111d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:53.641044 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0378473-b22f-4dde-8e77-5356a0d111d2 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:53.667383 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a3b4ad-ceda-45b2-b4af-f3b551ac239b None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:53.713020 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdf89ee-b81d-4821-abfc-7af3c1edcd61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:53.713685 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdf89ee-b81d-4821-abfc-7af3c1edcd61 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:53.713969 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdf89ee-b81d-4821-abfc-7af3c1edcd61 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:53.714243 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdf89ee-b81d-4821-abfc-7af3c1edcd61 None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:53.714965 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdf89ee-b81d-4821-abfc-7af3c1edcd61 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:53.715275 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdf89ee-b81d-4821-abfc-7af3c1edcd61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:53.716667 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdf89ee-b81d-4821-abfc-7af3c1edcd61 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:53.727854 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 365/735] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Jul 26 13:33:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:53.729135 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89fdcc6-8319-4b96-8bee-3137ae8f1e67 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:53.729534 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89fdcc6-8319-4b96-8bee-3137ae8f1e67 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:53.729830 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89fdcc6-8319-4b96-8bee-3137ae8f1e67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:53.767782 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 371/736] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Jul 26 13:33:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 13:33:53.769089 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0378473-b22f-4dde-8e77-5356a0d111d2 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:53.786625 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:53.865394 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-d89fdcc6-8319-4b96-8bee-3137ae8f1e67 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:33:53.866617 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-d89fdcc6-8319-4b96-8bee-3137ae8f1e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:33:53.888355 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0378473-b22f-4dde-8e77-5356a0d111d2 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:53.924540 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:53.931388 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f77ad7-50fb-454a-abbe-229290fe990a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:53.932358 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f77ad7-50fb-454a-abbe-229290fe990a None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:53.932801 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f77ad7-50fb-454a-abbe-229290fe990a None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:53.933217 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f77ad7-50fb-454a-abbe-229290fe990a None admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:53.934597 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f77ad7-50fb-454a-abbe-229290fe990a None admin] RBAC: Adding query filter params (name=service) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:53.935068 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f77ad7-50fb-454a-abbe-229290fe990a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:53.936431 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f77ad7-50fb-454a-abbe-229290fe990a None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:53.949931 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 372/737] 10.222.0.58 () {56 vars in 1103 bytes} [Tue Jul 26 13:33:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:53.949931 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f77ad7-50fb-454a-abbe-229290fe990a None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:53.955750 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 366/738] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:33:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 13:33:53.956816 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89fdcc6-8319-4b96-8bee-3137ae8f1e67 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:54.025042 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:54.057416 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f77ad7-50fb-454a-abbe-229290fe990a None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:54.076610 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89fdcc6-8319-4b96-8bee-3137ae8f1e67 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:54.098515 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5dae83-ac55-4196-849e-b560b98ceb66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:54.099159 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5dae83-ac55-4196-849e-b560b98ceb66 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:54.099532 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5dae83-ac55-4196-849e-b560b98ceb66 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:54.099811 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5dae83-ac55-4196-849e-b560b98ceb66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:54.110620 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5dae83-ac55-4196-849e-b560b98ceb66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:54.112068 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5dae83-ac55-4196-849e-b560b98ceb66 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:54.113550 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacafcf0-0db3-403a-9a72-4aac79083e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:54.114207 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacafcf0-0db3-403a-9a72-4aac79083e89 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:54.114490 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacafcf0-0db3-403a-9a72-4aac79083e89 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:54.114762 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacafcf0-0db3-403a-9a72-4aac79083e89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:54.125653 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacafcf0-0db3-403a-9a72-4aac79083e89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:54.127101 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacafcf0-0db3-403a-9a72-4aac79083e89 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:54.127208 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-8b5dae83-ac55-4196-849e-b560b98ceb66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:54.127208 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:54.127208 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:54.127208 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:54.127208 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:54.127208 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:54.127208 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:54.127208 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:54.127208 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:54.127208 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:54.127208 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:54.127208 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:54.127208 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 13:33:54.127208 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 13:33:54.127208 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 13:33:54.128226 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 13:33:54.128226 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:54.128226 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:54.128226 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:54.128226 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:54.128226 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:54.128226 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:54.128226 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:54.128226 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:54.128226 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:54.128226 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:54.128226 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:54.128226 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:54.128226 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:54.128226 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:54.128226 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:54.128226 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:54.129165 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 13:33:54.129165 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 13:33:54.129165 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:54.129165 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:33:54.129165 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 373/739] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:33:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:54.129452 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5dae83-ac55-4196-849e-b560b98ceb66 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:54.145300 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-aacafcf0-0db3-403a-9a72-4aac79083e89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:54.145300 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:54.145300 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:54.145300 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:54.145300 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:54.145300 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:54.145300 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:54.145300 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:54.145300 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:54.145300 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:54.145300 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:54.145300 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:54.145300 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 13:33:54.145300 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 13:33:54.145300 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 13:33:54.146462 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 13:33:54.146462 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:54.146462 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:54.146462 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:54.146462 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:54.146462 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:54.146462 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:54.146462 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:54.146462 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:54.146462 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:54.146462 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:54.146462 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:54.146462 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:54.146462 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:54.146462 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:54.146462 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:54.146462 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:54.147340 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 13:33:54.147340 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 13:33:54.147340 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:54.147340 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:33:54.147340 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 367/740] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:33:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:54.147340 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacafcf0-0db3-403a-9a72-4aac79083e89 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:54.180704 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:54.240975 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5dae83-ac55-4196-849e-b560b98ceb66 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:54.253449 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacafcf0-0db3-403a-9a72-4aac79083e89 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:54.264823 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:54.276782 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7646c879-a7a9-41c1-ae81-6b88be5616b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:54.277358 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7646c879-a7a9-41c1-ae81-6b88be5616b5 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:54.277685 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7646c879-a7a9-41c1-ae81-6b88be5616b5 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:54.277959 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7646c879-a7a9-41c1-ae81-6b88be5616b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:54.289294 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7646c879-a7a9-41c1-ae81-6b88be5616b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:54.291254 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7646c879-a7a9-41c1-ae81-6b88be5616b5 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:54.292363 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0375a4-5b47-4e2f-baa1-72a45e578057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:54.292987 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0375a4-5b47-4e2f-baa1-72a45e578057 None admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:54.293268 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0375a4-5b47-4e2f-baa1-72a45e578057 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:54.293570 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0375a4-5b47-4e2f-baa1-72a45e578057 None admin] PATH_INFO: `/v3/services` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:54.294261 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0375a4-5b47-4e2f-baa1-72a45e578057 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:54.295600 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0375a4-5b47-4e2f-baa1-72a45e578057 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:54.307849 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-7646c879-a7a9-41c1-ae81-6b88be5616b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:54.307849 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:54.307849 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:54.307849 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:54.307849 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:54.307849 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:54.307849 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:54.307849 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:54.307849 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:54.307849 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:54.307849 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:54.307849 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:54.307849 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 13:33:54.307849 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 13:33:54.307849 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 13:33:54.308773 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 13:33:54.308773 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:54.308773 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:54.308773 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:54.308773 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:54.308773 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:54.308773 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:54.308773 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:54.308773 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:54.308773 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:54.308773 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:54.308773 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:54.308773 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:54.308773 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:54.308773 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:54.308773 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:54.308773 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:54.309698 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 13:33:54.309698 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 13:33:54.309698 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:54.309698 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:33:54.309698 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 374/741] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:33:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:54.310229 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7989280c-6d2e-44c1-896c-ed4f8a976d87 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:54.310511 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7989280c-6d2e-44c1-896c-ed4f8a976d87 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:54.310785 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7989280c-6d2e-44c1-896c-ed4f8a976d87 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:54.312022 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 375/742] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:33:54 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:33:54.312702 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7989280c-6d2e-44c1-896c-ed4f8a976d87 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:54.318854 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 368/743] 10.222.0.58 () {60 vars in 1141 bytes} [Tue Jul 26 13:33:54 2022] POST /identity/v3/services => generated 251 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:33:54.320059 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0375a4-5b47-4e2f-baa1-72a45e578057 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:54.424430 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:54.445966 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7989280c-6d2e-44c1-896c-ed4f8a976d87 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:54.449628 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0375a4-5b47-4e2f-baa1-72a45e578057 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:54.482149 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0e5d82-6b85-4b48-b3af-fcb15e167700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:54.482772 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0e5d82-6b85-4b48-b3af-fcb15e167700 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:54.483053 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0e5d82-6b85-4b48-b3af-fcb15e167700 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:54.483324 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0e5d82-6b85-4b48-b3af-fcb15e167700 None admin] PATH_INFO: `/v3/domains` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:54.484014 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0e5d82-6b85-4b48-b3af-fcb15e167700 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:54.484315 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0e5d82-6b85-4b48-b3af-fcb15e167700 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:54.485682 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0e5d82-6b85-4b48-b3af-fcb15e167700 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:54.488988 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45df54cf-7dc6-4014-a1b5-6e16378be32c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:54.489918 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45df54cf-7dc6-4014-a1b5-6e16378be32c None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:54.490206 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45df54cf-7dc6-4014-a1b5-6e16378be32c None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:54.490488 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45df54cf-7dc6-4014-a1b5-6e16378be32c None admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:54.491379 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45df54cf-7dc6-4014-a1b5-6e16378be32c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:54.491497 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45df54cf-7dc6-4014-a1b5-6e16378be32c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:54.493710 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45df54cf-7dc6-4014-a1b5-6e16378be32c None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:54.500434 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 376/744] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Jul 26 13:33:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:54.501721 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0e5d82-6b85-4b48-b3af-fcb15e167700 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:54.506568 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 369/745] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Jul 26 13:33:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:54.507732 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf139b46-4e9b-4e29-b657-4aa88513de58 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:54.508037 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf139b46-4e9b-4e29-b657-4aa88513de58 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:54.508336 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf139b46-4e9b-4e29-b657-4aa88513de58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:54.530115 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-cf139b46-4e9b-4e29-b657-4aa88513de58 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:33:54.531494 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-cf139b46-4e9b-4e29-b657-4aa88513de58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:33:54.541122 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:54.612226 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 370/746] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:33:54 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 13:33:54.621688 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf139b46-4e9b-4e29-b657-4aa88513de58 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:54.621775 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0e5d82-6b85-4b48-b3af-fcb15e167700 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:54.658924 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e067c1-2fad-42ea-aa70-58abfdb40fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:54.659555 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e067c1-2fad-42ea-aa70-58abfdb40fe0 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:54.659836 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e067c1-2fad-42ea-aa70-58abfdb40fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:54.660106 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e067c1-2fad-42ea-aa70-58abfdb40fe0 None admin] PATH_INFO: `/v3/domains` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:54.660797 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e067c1-2fad-42ea-aa70-58abfdb40fe0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:54.661101 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e067c1-2fad-42ea-aa70-58abfdb40fe0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:54.662490 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e067c1-2fad-42ea-aa70-58abfdb40fe0 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:54.680188 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 377/747] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Jul 26 13:33:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:54.680188 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e067c1-2fad-42ea-aa70-58abfdb40fe0 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:54.705887 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:54.730721 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf139b46-4e9b-4e29-b657-4aa88513de58 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:54.766746 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c443cbf2-79e1-4316-9b2c-73ad9aae0dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:54.768293 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c443cbf2-79e1-4316-9b2c-73ad9aae0dc1 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:54.769179 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c443cbf2-79e1-4316-9b2c-73ad9aae0dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:54.769831 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c443cbf2-79e1-4316-9b2c-73ad9aae0dc1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:54.780796 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c443cbf2-79e1-4316-9b2c-73ad9aae0dc1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:54.783029 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:54.783289 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c443cbf2-79e1-4316-9b2c-73ad9aae0dc1 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:54.793831 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e067c1-2fad-42ea-aa70-58abfdb40fe0 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:54.799608 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-c443cbf2-79e1-4316-9b2c-73ad9aae0dc1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 26 13:33:54.799608 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:54.799608 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:54.799608 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:54.799608 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:54.799608 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:54.799608 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:54.799608 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:54.799608 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:54.799608 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:54.799608 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:54.799608 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:54.799608 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 13:33:54.799608 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 13:33:54.799608 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 13:33:54.799608 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 13:33:54.800570 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:54.800570 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:54.800570 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 13:33:54.800570 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 13:33:54.800570 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 13:33:54.800570 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 13:33:54.800570 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:54.800570 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:54.800570 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:54.800570 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:54.800570 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:54.800570 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:54.800570 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:54.800570 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:54.800570 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:54.800570 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:54.800570 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:54.800570 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:54.801594 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:54.801594 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:54.801594 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 13:33:54.801594 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 13:33:54.801594 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 13:33:54.801594 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 13:33:54.801594 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 13:33:54.801594 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 13:33:54.801594 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 26 13:33:54.801594 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:33:54.805505 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 371/748] 10.222.0.58 () {56 vars in 1093 bytes} [Tue Jul 26 13:33:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:54.806484 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c443cbf2-79e1-4316-9b2c-73ad9aae0dc1 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:54.829407 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceb28a5-ecfd-4a50-8e85-c7b9c5d5d7f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:54.830098 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceb28a5-ecfd-4a50-8e85-c7b9c5d5d7f3 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:54.830380 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceb28a5-ecfd-4a50-8e85-c7b9c5d5d7f3 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:54.830655 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceb28a5-ecfd-4a50-8e85-c7b9c5d5d7f3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:54.838559 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ceb28a5-ecfd-4a50-8e85-c7b9c5d5d7f3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:54.840067 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ceb28a5-ecfd-4a50-8e85-c7b9c5d5d7f3 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:54.852907 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-2ceb28a5-ecfd-4a50-8e85-c7b9c5d5d7f3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 13:33:54.852907 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:54.852907 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:54.852907 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:54.852907 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:54.852907 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:54.852907 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:54.852907 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:54.852907 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:54.852907 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:54.852907 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:54.852907 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:54.852907 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 13:33:54.852907 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 13:33:54.852907 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 13:33:54.853891 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 13:33:54.853891 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:54.853891 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:54.853891 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:54.853891 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:54.853891 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:54.853891 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:54.853891 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:54.853891 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:54.853891 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:54.853891 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:54.853891 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:54.853891 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:54.853891 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:54.853891 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:54.853891 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:54.853891 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:54.854791 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 13:33:54.854791 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 13:33:54.854791 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 13:33:54.854791 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 13:33:54.854791 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 13:33:54.854791 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 13:33:54.854791 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 13:33:54.854791 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:33:54.855772 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 378/749] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Jul 26 13:33:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:54.856580 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceb28a5-ecfd-4a50-8e85-c7b9c5d5d7f3 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:54.915071 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c443cbf2-79e1-4316-9b2c-73ad9aae0dc1 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:54.950842 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14dab90-2ab7-42f1-8c60-10215ace668a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:54.951567 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14dab90-2ab7-42f1-8c60-10215ace668a None admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:54.951850 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14dab90-2ab7-42f1-8c60-10215ace668a None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:54.952246 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14dab90-2ab7-42f1-8c60-10215ace668a None admin] PATH_INFO: `/v3/services` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:54.952940 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14dab90-2ab7-42f1-8c60-10215ace668a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:54.954350 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14dab90-2ab7-42f1-8c60-10215ace668a None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:54.958089 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceb28a5-ecfd-4a50-8e85-c7b9c5d5d7f3 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:54.964033 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:54.985672 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 372/750] 10.222.0.58 () {60 vars in 1141 bytes} [Tue Jul 26 13:33:54 2022] POST /identity/v3/services => generated 240 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:33:54.987195 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14dab90-2ab7-42f1-8c60-10215ace668a None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:55.014378 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f43870-9078-406d-abdb-dfb1dcbe1ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:55.015408 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f43870-9078-406d-abdb-dfb1dcbe1ae7 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:55.015989 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f43870-9078-406d-abdb-dfb1dcbe1ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:55.016513 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f43870-9078-406d-abdb-dfb1dcbe1ae7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:55.026158 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f43870-9078-406d-abdb-dfb1dcbe1ae7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:55.028284 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f43870-9078-406d-abdb-dfb1dcbe1ae7 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:55.050798 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-25f43870-9078-406d-abdb-dfb1dcbe1ae7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 13:33:55.050798 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:55.050798 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:55.050798 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:55.050798 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:55.050798 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:55.050798 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:55.050798 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:55.050798 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:55.050798 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:55.050798 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:55.050798 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:55.050798 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 13:33:55.050798 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 13:33:55.050798 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 13:33:55.050798 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 13:33:55.051847 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:55.051847 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:55.051847 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 13:33:55.051847 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 13:33:55.051847 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 13:33:55.051847 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 13:33:55.051847 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:55.051847 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:55.051847 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:55.051847 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:55.051847 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:55.051847 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:55.051847 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:55.051847 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:55.051847 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:55.051847 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:55.051847 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:55.051847 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:55.052842 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:55.052842 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:55.052842 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 13:33:55.052842 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 13:33:55.052842 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 13:33:55.052842 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 13:33:55.052842 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 13:33:55.052842 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 13:33:55.052842 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 13:33:55.052842 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:33:55.060059 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 379/751] 10.222.0.58 () {56 vars in 1117 bytes} [Tue Jul 26 13:33:54 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:55.060059 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f43870-9078-406d-abdb-dfb1dcbe1ae7 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:55.073313 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:55.147259 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14dab90-2ab7-42f1-8c60-10215ace668a None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:55.176659 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f43870-9078-406d-abdb-dfb1dcbe1ae7 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:55.195361 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9069a15-f47c-4a4d-972a-6742ce95f026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:55.196354 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9069a15-f47c-4a4d-972a-6742ce95f026 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:55.196822 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9069a15-f47c-4a4d-972a-6742ce95f026 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:55.197306 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9069a15-f47c-4a4d-972a-6742ce95f026 None admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:55.198402 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9069a15-f47c-4a4d-972a-6742ce95f026 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:55.198915 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9069a15-f47c-4a4d-972a-6742ce95f026 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:55.200562 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9069a15-f47c-4a4d-972a-6742ce95f026 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:55.220388 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 373/752] 10.222.0.58 () {56 vars in 1133 bytes} [Tue Jul 26 13:33:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:55.221241 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9069a15-f47c-4a4d-972a-6742ce95f026 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:55.222085 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13eb1d85-0490-417c-b5b4-abf58e6aa5da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:55.223620 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eb1d85-0490-417c-b5b4-abf58e6aa5da None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:55.223978 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eb1d85-0490-417c-b5b4-abf58e6aa5da None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:55.224328 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eb1d85-0490-417c-b5b4-abf58e6aa5da None admin] PATH_INFO: `/v3/projects` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:55.225303 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13eb1d85-0490-417c-b5b4-abf58e6aa5da None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:55.226059 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13eb1d85-0490-417c-b5b4-abf58e6aa5da None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:55.228172 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13eb1d85-0490-417c-b5b4-abf58e6aa5da None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:55.246932 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 380/753] 10.222.0.58 () {56 vars in 1148 bytes} [Tue Jul 26 13:33:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:55.247792 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13eb1d85-0490-417c-b5b4-abf58e6aa5da None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:55.298030 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:55.331413 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:55.362989 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9069a15-f47c-4a4d-972a-6742ce95f026 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:55.370511 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13eb1d85-0490-417c-b5b4-abf58e6aa5da None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:55.404076 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982c0f70-6b8f-453f-b9e9-2c93c61953e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:55.404874 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982c0f70-6b8f-453f-b9e9-2c93c61953e6 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:55.405455 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982c0f70-6b8f-453f-b9e9-2c93c61953e6 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:55.406201 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982c0f70-6b8f-453f-b9e9-2c93c61953e6 None admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:55.407445 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982c0f70-6b8f-453f-b9e9-2c93c61953e6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:55.407881 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982c0f70-6b8f-453f-b9e9-2c93c61953e6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:55.409465 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982c0f70-6b8f-453f-b9e9-2c93c61953e6 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:55.410631 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4804547a-1c4c-4f57-bd42-ee05b1fd76dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:55.411388 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4804547a-1c4c-4f57-bd42-ee05b1fd76dd None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:55.411673 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4804547a-1c4c-4f57-bd42-ee05b1fd76dd None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:55.411948 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4804547a-1c4c-4f57-bd42-ee05b1fd76dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:55.428389 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4804547a-1c4c-4f57-bd42-ee05b1fd76dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:55.429886 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4804547a-1c4c-4f57-bd42-ee05b1fd76dd None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:55.432331 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 374/754] 10.222.0.58 () {56 vars in 1149 bytes} [Tue Jul 26 13:33:55 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:55.433189 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982c0f70-6b8f-453f-b9e9-2c93c61953e6 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:55.450870 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-4804547a-1c4c-4f57-bd42-ee05b1fd76dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:55.450870 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:55.450870 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:55.450870 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:55.450870 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:55.450870 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:55.450870 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:55.450870 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:55.450870 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:55.450870 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:55.450870 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:55.450870 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:55.450870 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 13:33:55.450870 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 13:33:55.450870 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 13:33:55.452840 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 13:33:55.452840 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:55.452840 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:55.452840 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:55.452840 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:55.452840 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:55.452840 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:55.452840 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:55.452840 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:55.452840 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:55.452840 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:55.452840 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:55.452840 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:55.452840 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:55.452840 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:55.452840 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:55.452840 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:55.454815 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 13:33:55.454815 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 13:33:55.454815 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:55.454815 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:33:55.458009 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 381/755] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:33:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:55.458009 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4804547a-1c4c-4f57-bd42-ee05b1fd76dd None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:55.562068 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982c0f70-6b8f-453f-b9e9-2c93c61953e6 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:55.572951 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4804547a-1c4c-4f57-bd42-ee05b1fd76dd None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:55.586979 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:55.593823 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:55.599248 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0c0d25-dbe7-4267-ad38-dc324d798288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:55.600215 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0c0d25-dbe7-4267-ad38-dc324d798288 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:55.600502 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0c0d25-dbe7-4267-ad38-dc324d798288 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:55.600777 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0c0d25-dbe7-4267-ad38-dc324d798288 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:55.601760 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0c0d25-dbe7-4267-ad38-dc324d798288 None admin] RBAC: Adding query filter params (role.id=6e0201a9a6484f7d8b112b1eafff70bd, scope.project.id=1e8bb4385c204210a1bc77b1f0d654f3, user.id=3ab2819bac844163ae4305e4f2186605) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:55.602937 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0c0d25-dbe7-4267-ad38-dc324d798288 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:55.604656 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0c0d25-dbe7-4267-ad38-dc324d798288 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:55.611354 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4b52f7-cae9-4616-a5fe-025cc9d5ab0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:55.611788 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 375/756] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Jul 26 13:33:55 2022] GET /identity/v3/role_assignments?user.id=3ab2819bac844163ae4305e4f2186605&scope.project.id=1e8bb4385c204210a1bc77b1f0d654f3&role.id=6e0201a9a6484f7d8b112b1eafff70bd => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:55.612193 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4b52f7-cae9-4616-a5fe-025cc9d5ab0c None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:55.612653 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4b52f7-cae9-4616-a5fe-025cc9d5ab0c None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:55.612763 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0c0d25-dbe7-4267-ad38-dc324d798288 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:55.613103 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4b52f7-cae9-4616-a5fe-025cc9d5ab0c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:55.689499 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4b52f7-cae9-4616-a5fe-025cc9d5ab0c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:55.691118 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4b52f7-cae9-4616-a5fe-025cc9d5ab0c None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:55.754380 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0c0d25-dbe7-4267-ad38-dc324d798288 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:55.811442 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c831f2-721e-4de6-9484-460a50c51566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:55.812153 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c831f2-721e-4de6-9484-460a50c51566 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:55.812378 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c831f2-721e-4de6-9484-460a50c51566 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:55.812653 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c831f2-721e-4de6-9484-460a50c51566 None admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:55.813372 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c831f2-721e-4de6-9484-460a50c51566 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:55.813713 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c831f2-721e-4de6-9484-460a50c51566 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:55.815127 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c831f2-721e-4de6-9484-460a50c51566 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:55.840765 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 376/757] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Jul 26 13:33:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:55.847228 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c831f2-721e-4de6-9484-460a50c51566 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:55.857344 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 382/758] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Jul 26 13:33:55 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 13:33:55.863303 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4b52f7-cae9-4616-a5fe-025cc9d5ab0c None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:55.894523 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:55.924576 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:55.998593 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c831f2-721e-4de6-9484-460a50c51566 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:56.051111 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4b52f7-cae9-4616-a5fe-025cc9d5ab0c None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:56.053868 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b87d38-2859-4826-b767-095053813e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:56.054533 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b87d38-2859-4826-b767-095053813e1d None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:56.054813 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b87d38-2859-4826-b767-095053813e1d None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:56.055128 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b87d38-2859-4826-b767-095053813e1d None admin] PATH_INFO: `/v3/projects/service` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:56.070547 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b87d38-2859-4826-b767-095053813e1d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:56.072478 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b87d38-2859-4826-b767-095053813e1d None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:56.091170 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-43b87d38-2859-4826-b767-095053813e1d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 13:33:56.091170 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:56.091170 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:56.091170 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:56.091170 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:56.091170 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:56.091170 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:56.091170 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:56.091170 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:56.091170 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:56.091170 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:56.091170 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:56.091170 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 13:33:56.091170 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 13:33:56.091170 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 13:33:56.092523 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 13:33:56.092523 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:56.092523 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:56.092523 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:56.092523 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:56.092523 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:56.092523 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:56.092523 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:56.092523 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:56.092523 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:56.092523 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:56.092523 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:56.092523 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:56.092523 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:56.092523 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:56.092523 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:56.092523 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:56.094404 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 13:33:56.094404 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 13:33:56.094404 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 13:33:56.094404 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 13:33:56.094404 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 13:33:56.094404 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 13:33:56.094404 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 13:33:56.094404 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:33:56.094404 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 377/759] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Jul 26 13:33:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 13:33:56.101132 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b87d38-2859-4826-b767-095053813e1d None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:56.125197 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d75ae53-3eef-46da-b781-75d3fbbc7a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:56.125879 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d75ae53-3eef-46da-b781-75d3fbbc7a27 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:56.126157 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d75ae53-3eef-46da-b781-75d3fbbc7a27 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:56.126426 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d75ae53-3eef-46da-b781-75d3fbbc7a27 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:56.140436 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d75ae53-3eef-46da-b781-75d3fbbc7a27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:56.141923 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d75ae53-3eef-46da-b781-75d3fbbc7a27 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:56.163866 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-3d75ae53-3eef-46da-b781-75d3fbbc7a27 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:56.163866 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:56.163866 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:56.163866 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:56.163866 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:56.163866 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:56.163866 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:56.163866 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:56.163866 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:56.163866 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:56.163866 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:56.163866 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:56.163866 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 13:33:56.163866 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 13:33:56.163866 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 13:33:56.165715 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 13:33:56.165715 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:56.165715 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:56.165715 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:56.165715 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:56.165715 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:56.165715 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:56.165715 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:56.165715 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:56.165715 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:56.165715 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:56.165715 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:56.165715 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:56.165715 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:56.165715 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:56.165715 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:56.165715 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:56.166602 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 13:33:56.166602 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 13:33:56.166602 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:56.166602 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:33:56.167250 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 383/760] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:33:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 13:33:56.173531 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d75ae53-3eef-46da-b781-75d3fbbc7a27 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:56.291464 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:56.298548 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b87d38-2859-4826-b767-095053813e1d None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:56.326841 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:56.358818 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1c2f8f-e07e-4e75-9887-24ab84818846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:56.359569 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1c2f8f-e07e-4e75-9887-24ab84818846 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:56.359869 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1c2f8f-e07e-4e75-9887-24ab84818846 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:56.360150 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1c2f8f-e07e-4e75-9887-24ab84818846 None admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:56.360917 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1c2f8f-e07e-4e75-9887-24ab84818846 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:56.361233 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1c2f8f-e07e-4e75-9887-24ab84818846 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:56.366933 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1c2f8f-e07e-4e75-9887-24ab84818846 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:56.372309 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d75ae53-3eef-46da-b781-75d3fbbc7a27 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:56.393829 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 378/761] 10.222.0.58 () {56 vars in 1148 bytes} [Tue Jul 26 13:33:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:56.400331 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1c2f8f-e07e-4e75-9887-24ab84818846 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:56.445006 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb17f2d0-c54a-49e7-adbd-cc0f281cde9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:56.445722 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb17f2d0-c54a-49e7-adbd-cc0f281cde9f None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:56.446006 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb17f2d0-c54a-49e7-adbd-cc0f281cde9f None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:56.446282 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb17f2d0-c54a-49e7-adbd-cc0f281cde9f None admin] PATH_INFO: `/v3/domains` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:56.446987 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb17f2d0-c54a-49e7-adbd-cc0f281cde9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:56.447292 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb17f2d0-c54a-49e7-adbd-cc0f281cde9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:56.448673 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb17f2d0-c54a-49e7-adbd-cc0f281cde9f None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:56.475582 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 384/762] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Jul 26 13:33:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:56.476468 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb17f2d0-c54a-49e7-adbd-cc0f281cde9f None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:56.589125 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1c2f8f-e07e-4e75-9887-24ab84818846 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:56.598595 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb17f2d0-c54a-49e7-adbd-cc0f281cde9f None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:56.625814 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ef3b9a-c209-426f-8285-3bf3cebe5e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:56.626809 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef3b9a-c209-426f-8285-3bf3cebe5e6f None admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:56.627360 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef3b9a-c209-426f-8285-3bf3cebe5e6f None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:56.627837 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef3b9a-c209-426f-8285-3bf3cebe5e6f None admin] PATH_INFO: `/v3/projects/1e8bb4385c204210a1bc77b1f0d654f3/users/c69161cd520b4b29bf50f3bf2f1c033b/roles/6e0201a9a6484f7d8b112b1eafff70bd` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:56.630029 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:56.631209 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ef3b9a-c209-426f-8285-3bf3cebe5e6f None admin] RBAC: Authorizing `identity:create_grant(project_id=1e8bb4385c204210a1bc77b1f0d654f3, user_id=c69161cd520b4b29bf50f3bf2f1c033b, role_id=6e0201a9a6484f7d8b112b1eafff70bd)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:56.632912 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ef3b9a-c209-426f-8285-3bf3cebe5e6f None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:56.636155 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046bf524-4886-4883-9758-ce64ace12001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:56.637169 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046bf524-4886-4883-9758-ce64ace12001 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:56.637796 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046bf524-4886-4883-9758-ce64ace12001 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:56.638267 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046bf524-4886-4883-9758-ce64ace12001 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:56.646774 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046bf524-4886-4883-9758-ce64ace12001 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:56.648348 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046bf524-4886-4883-9758-ce64ace12001 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:56.652596 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 379/763] 10.222.0.58 () {58 vars in 1439 bytes} [Tue Jul 26 13:33:56 2022] PUT /identity/v3/projects/1e8bb4385c204210a1bc77b1f0d654f3/users/c69161cd520b4b29bf50f3bf2f1c033b/roles/6e0201a9a6484f7d8b112b1eafff70bd => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 13:33:56.654397 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e1d95c-7e89-412f-bdf7-d0c57aa862fe None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:56.655036 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e1d95c-7e89-412f-bdf7-d0c57aa862fe None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:56.655650 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e1d95c-7e89-412f-bdf7-d0c57aa862fe None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:56.657771 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 380/764] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:33:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:33:56.666537 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-046bf524-4886-4883-9758-ce64ace12001 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 13:33:56.666537 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:56.666537 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:56.666537 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:56.666537 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:56.666537 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:56.666537 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:56.666537 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:56.666537 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:56.666537 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:56.666537 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:56.666537 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:56.666537 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 13:33:56.666537 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 13:33:56.666537 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 13:33:56.667495 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 13:33:56.667495 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:56.667495 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:56.667495 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:56.667495 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:56.667495 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:56.667495 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:56.667495 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:56.667495 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:56.667495 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:56.667495 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:56.667495 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:56.667495 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:56.667495 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:56.667495 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:56.667495 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:56.667495 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:56.668367 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 13:33:56.668367 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 13:33:56.668367 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 13:33:56.668367 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 13:33:56.668367 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 13:33:56.668367 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 13:33:56.668367 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 13:33:56.668367 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:33:56.668792 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b208986a-bf71-4887-8213-eb834ef6782e None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:56.668792 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b208986a-bf71-4887-8213-eb834ef6782e None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:56.668792 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b208986a-bf71-4887-8213-eb834ef6782e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:56.671446 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 385/765] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Jul 26 13:33:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:56.678869 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046bf524-4886-4883-9758-ce64ace12001 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:56.707015 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-b208986a-bf71-4887-8213-eb834ef6782e None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:33:56.708382 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-b208986a-bf71-4887-8213-eb834ef6782e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:33:56.747054 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:56.834396 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046bf524-4886-4883-9758-ce64ace12001 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:56.873955 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 381/766] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:33:56 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 13:33:56.880896 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7ac62f-7d67-40a5-852c-89f458239516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:56.881864 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7ac62f-7d67-40a5-852c-89f458239516 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:56.882357 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7ac62f-7d67-40a5-852c-89f458239516 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:56.882924 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7ac62f-7d67-40a5-852c-89f458239516 None admin] PATH_INFO: `/v3/projects` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:56.884420 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7ac62f-7d67-40a5-852c-89f458239516 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:56.884946 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7ac62f-7d67-40a5-852c-89f458239516 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:56.886611 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7ac62f-7d67-40a5-852c-89f458239516 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:56.901583 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b208986a-bf71-4887-8213-eb834ef6782e None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:56.903840 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 386/767] 10.222.0.58 () {56 vars in 1148 bytes} [Tue Jul 26 13:33:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:56.909223 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7ac62f-7d67-40a5-852c-89f458239516 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:56.970453 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:57.025845 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7ac62f-7d67-40a5-852c-89f458239516 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:57.030070 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b208986a-bf71-4887-8213-eb834ef6782e None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:57.064725 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e33a8f-d765-46ce-8cb0-782d6ba8f63d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:57.065730 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e33a8f-d765-46ce-8cb0-782d6ba8f63d None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:57.066226 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e33a8f-d765-46ce-8cb0-782d6ba8f63d None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:57.066703 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e33a8f-d765-46ce-8cb0-782d6ba8f63d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:57.067719 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e33a8f-d765-46ce-8cb0-782d6ba8f63d None admin] RBAC: Adding query filter params (role.id=17e0f3c73a8a4a54b3e7b78878dfbf09, scope.project.id=1e8bb4385c204210a1bc77b1f0d654f3, user.id=b64bd7b183d24d8c9037b261c7884810) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:57.068254 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e33a8f-d765-46ce-8cb0-782d6ba8f63d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:57.069953 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e33a8f-d765-46ce-8cb0-782d6ba8f63d None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:57.076999 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226bb8a3-4f4a-4f77-b850-48cf8437b8f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:57.079103 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226bb8a3-4f4a-4f77-b850-48cf8437b8f3 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:57.080338 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226bb8a3-4f4a-4f77-b850-48cf8437b8f3 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:57.080987 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226bb8a3-4f4a-4f77-b850-48cf8437b8f3 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:57.082797 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 387/768] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Jul 26 13:33:56 2022] GET /identity/v3/role_assignments?user.id=b64bd7b183d24d8c9037b261c7884810&scope.project.id=1e8bb4385c204210a1bc77b1f0d654f3&role.id=17e0f3c73a8a4a54b3e7b78878dfbf09 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:57.083833 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226bb8a3-4f4a-4f77-b850-48cf8437b8f3 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:57.085429 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226bb8a3-4f4a-4f77-b850-48cf8437b8f3 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:57.095682 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-226bb8a3-4f4a-4f77-b850-48cf8437b8f3 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 26 13:33:57.095682 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:57.095682 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 13:33:57.095682 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 13:33:57.095682 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 13:33:57.095682 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 13:33:57.095682 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 13:33:57.095682 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 13:33:57.095682 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 26 13:33:57.095682 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:33:57.095682 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 13:33:57.095682 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:33:57.095682 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:57.095682 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:57.095682 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:57.095682 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:57.096860 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:57.096860 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:57.096860 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:57.096860 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:57.096860 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:57.096860 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:57.096860 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:57.096860 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 13:33:57.096860 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 13:33:57.096860 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 13:33:57.096860 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 13:33:57.096860 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:57.096860 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:57.096860 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:57.096860 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:57.096860 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:57.097813 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:57.097813 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:57.097813 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:57.097813 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:57.097813 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:57.097813 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:57.097813 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:57.097813 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:57.097813 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:57.097813 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:57.097813 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:57.097813 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 13:33:57.097813 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 13:33:57.097813 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 26 13:33:57.097813 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:33:57.104820 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 382/769] 10.222.0.58 () {56 vars in 1129 bytes} [Tue Jul 26 13:33:56 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 13:33:57.106417 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e33a8f-d765-46ce-8cb0-782d6ba8f63d None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:57.260258 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e33a8f-d765-46ce-8cb0-782d6ba8f63d None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:57.277416 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:57.286009 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d2f56c-957c-4b21-8627-3d95a5e7fc46 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:57.286972 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d2f56c-957c-4b21-8627-3d95a5e7fc46 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:57.287412 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d2f56c-957c-4b21-8627-3d95a5e7fc46 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:57.293683 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 383/770] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:33:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:33:57.295839 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde23079-acbc-4ca0-8ed7-6fa284104e0c None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:57.296252 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde23079-acbc-4ca0-8ed7-6fa284104e0c None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:57.296531 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde23079-acbc-4ca0-8ed7-6fa284104e0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:57.303184 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4143600-96f9-452a-9719-a1e57bdd72ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:57.304393 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4143600-96f9-452a-9719-a1e57bdd72ce None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:57.304875 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4143600-96f9-452a-9719-a1e57bdd72ce None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:57.307124 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4143600-96f9-452a-9719-a1e57bdd72ce None admin] PATH_INFO: `/v3/services` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:57.308037 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4143600-96f9-452a-9719-a1e57bdd72ce None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:57.308540 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4143600-96f9-452a-9719-a1e57bdd72ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:57.310132 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4143600-96f9-452a-9719-a1e57bdd72ce None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:57.316376 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 388/771] 10.222.0.58 () {56 vars in 1124 bytes} [Tue Jul 26 13:33:57 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:57.316851 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-bde23079-acbc-4ca0-8ed7-6fa284104e0c None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:33:57.318224 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-bde23079-acbc-4ca0-8ed7-6fa284104e0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:33:57.322888 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4143600-96f9-452a-9719-a1e57bdd72ce None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:57.329695 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:57.464749 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4143600-96f9-452a-9719-a1e57bdd72ce None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:57.493576 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 384/772] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:33:57 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 13:33:57.515660 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d133f61-442e-4036-bcf7-d4ffd5e30619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:57.516618 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d133f61-442e-4036-bcf7-d4ffd5e30619 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:57.517116 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d133f61-442e-4036-bcf7-d4ffd5e30619 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:57.517630 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d133f61-442e-4036-bcf7-d4ffd5e30619 None admin] PATH_INFO: `/v3/services` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:57.518559 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d133f61-442e-4036-bcf7-d4ffd5e30619 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:57.519072 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d133f61-442e-4036-bcf7-d4ffd5e30619 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:57.520853 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d133f61-442e-4036-bcf7-d4ffd5e30619 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:57.532966 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde23079-acbc-4ca0-8ed7-6fa284104e0c None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:57.533369 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 389/773] 10.222.0.58 () {56 vars in 1124 bytes} [Tue Jul 26 13:33:57 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:33:57.541365 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d133f61-442e-4036-bcf7-d4ffd5e30619 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:57.595123 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:57.667310 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d133f61-442e-4036-bcf7-d4ffd5e30619 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:57.708115 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2cb542-8fe7-425c-864d-c6afe3c29e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:57.709162 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2cb542-8fe7-425c-864d-c6afe3c29e81 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:57.709670 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2cb542-8fe7-425c-864d-c6afe3c29e81 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:57.710144 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2cb542-8fe7-425c-864d-c6afe3c29e81 None admin] PATH_INFO: `/v3/services/network` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:57.710972 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2cb542-8fe7-425c-864d-c6afe3c29e81 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:57.713062 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2cb542-8fe7-425c-864d-c6afe3c29e81 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:57.728185 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-0a2cb542-8fe7-425c-864d-c6afe3c29e81 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 26 13:33:57.728185 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:57.728185 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 13:33:57.728185 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 13:33:57.728185 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 13:33:57.728185 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 13:33:57.728185 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 13:33:57.728185 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 13:33:57.728185 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 26 13:33:57.728185 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:33:57.728185 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 13:33:57.728185 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:33:57.728185 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:57.728185 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:57.728185 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:57.728185 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:57.728185 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:57.729218 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:57.729218 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:57.729218 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:57.729218 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:57.729218 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:57.729218 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:57.729218 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 13:33:57.729218 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 13:33:57.729218 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 13:33:57.729218 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 13:33:57.729218 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:57.729218 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:57.729218 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:57.729218 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:57.729218 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:57.729218 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:57.730090 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:57.730090 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:57.730090 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:57.730090 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:57.730090 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:57.730090 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:57.730090 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:57.730090 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:57.730090 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:57.730090 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:57.730090 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 13:33:57.730090 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 13:33:57.730090 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 26 13:33:57.730090 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:33:57.733919 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 390/774] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Jul 26 13:33:57 2022] GET /identity/v3/services/network => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 13:33:57.740352 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2cb542-8fe7-425c-864d-c6afe3c29e81 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:57.795675 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde23079-acbc-4ca0-8ed7-6fa284104e0c None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:57.868414 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2cb542-8fe7-425c-864d-c6afe3c29e81 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:57.876457 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6557df-a784-4247-b318-89d525882aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:57.876932 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6557df-a784-4247-b318-89d525882aa3 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:57.877229 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6557df-a784-4247-b318-89d525882aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:57.877522 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6557df-a784-4247-b318-89d525882aa3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:57.878304 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6557df-a784-4247-b318-89d525882aa3 None admin] RBAC: Adding query filter params (interface=public, service_id=183c1ff1f48a4131953b6683cf611ae5, region_id=RegionOne) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:57.878628 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6557df-a784-4247-b318-89d525882aa3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:57.879839 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:57.881184 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6557df-a784-4247-b318-89d525882aa3 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:57.902785 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 385/775] 10.222.0.58 () {56 vars in 1251 bytes} [Tue Jul 26 13:33:57 2022] GET /identity/v3/endpoints?service_id=183c1ff1f48a4131953b6683cf611ae5&interface=public®ion_id=RegionOne => generated 194 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:57.910534 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6557df-a784-4247-b318-89d525882aa3 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:57.927443 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:57.945865 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89013a4e-8126-4c64-afbd-6a213d950023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:57.947872 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89013a4e-8126-4c64-afbd-6a213d950023 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:57.948804 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89013a4e-8126-4c64-afbd-6a213d950023 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:57.950134 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89013a4e-8126-4c64-afbd-6a213d950023 None admin] PATH_INFO: `/v3/services` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:57.951262 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89013a4e-8126-4c64-afbd-6a213d950023 None admin] RBAC: Adding query filter params (name=network) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:57.954643 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89013a4e-8126-4c64-afbd-6a213d950023 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:57.956294 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89013a4e-8126-4c64-afbd-6a213d950023 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:57.965229 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 391/776] 10.222.0.58 () {56 vars in 1112 bytes} [Tue Jul 26 13:33:57 2022] GET /identity/v3/services?name=network => generated 124 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:33:57.971491 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89013a4e-8126-4c64-afbd-6a213d950023 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:58.147473 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89013a4e-8126-4c64-afbd-6a213d950023 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:58.154661 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6557df-a784-4247-b318-89d525882aa3 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:58.203178 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4984b22c-596e-429e-89d8-04741ab6f66d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:58.203830 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4984b22c-596e-429e-89d8-04741ab6f66d None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:58.204113 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4984b22c-596e-429e-89d8-04741ab6f66d None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:58.204392 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4984b22c-596e-429e-89d8-04741ab6f66d None admin] PATH_INFO: `/v3/services` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:58.205077 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4984b22c-596e-429e-89d8-04741ab6f66d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:58.206574 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4984b22c-596e-429e-89d8-04741ab6f66d None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:58.211915 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2043e09a-95cf-4503-89b0-79208f7a5802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:58.212934 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2043e09a-95cf-4503-89b0-79208f7a5802 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:58.213448 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2043e09a-95cf-4503-89b0-79208f7a5802 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:58.213963 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 386/777] 10.222.0.58 () {56 vars in 1087 bytes} [Tue Jul 26 13:33:57 2022] GET /identity/v3/services => generated 772 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:58.214238 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2043e09a-95cf-4503-89b0-79208f7a5802 None admin] PATH_INFO: `/v3/services` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:58.215137 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2043e09a-95cf-4503-89b0-79208f7a5802 None admin] RBAC: Adding query filter params (type=network) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:58.215324 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd111690-8339-4ff7-b41a-75d3bc707fb4 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:58.215606 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd111690-8339-4ff7-b41a-75d3bc707fb4 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:58.215884 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd111690-8339-4ff7-b41a-75d3bc707fb4 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:58.216240 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2043e09a-95cf-4503-89b0-79208f7a5802 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:58.217912 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2043e09a-95cf-4503-89b0-79208f7a5802 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:58.221783 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 387/778] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:33:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:33:58.223273 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00ebb12-feee-4abc-a3aa-bf870eb10315 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:58.223568 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00ebb12-feee-4abc-a3aa-bf870eb10315 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:58.223847 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00ebb12-feee-4abc-a3aa-bf870eb10315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:58.225950 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 392/779] 10.222.0.58 () {56 vars in 1112 bytes} [Tue Jul 26 13:33:57 2022] GET /identity/v3/services?type=network => generated 350 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:58.234644 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2043e09a-95cf-4503-89b0-79208f7a5802 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:58.267194 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-e00ebb12-feee-4abc-a3aa-bf870eb10315 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:33:58.268423 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-e00ebb12-feee-4abc-a3aa-bf870eb10315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:33:58.309821 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:58.394670 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:58.402180 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 388/780] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:33:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 13:33:58.420790 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2043e09a-95cf-4503-89b0-79208f7a5802 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:58.442429 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00ebb12-feee-4abc-a3aa-bf870eb10315 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:58.470905 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3809baa9-c0ef-4b08-a97f-e22424e5c7aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:58.472883 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3809baa9-c0ef-4b08-a97f-e22424e5c7aa None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:58.473567 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3809baa9-c0ef-4b08-a97f-e22424e5c7aa None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:58.474139 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3809baa9-c0ef-4b08-a97f-e22424e5c7aa None admin] PATH_INFO: `/v3/endpoints` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:58.475339 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3809baa9-c0ef-4b08-a97f-e22424e5c7aa None admin] RBAC: Adding query filter params (interface=public, service_id=fcf30d1c2e004d25ad5a47df9d25357d, region_id=RegionOne) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:58.475859 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3809baa9-c0ef-4b08-a97f-e22424e5c7aa None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:58.478210 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3809baa9-c0ef-4b08-a97f-e22424e5c7aa None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:58.489885 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 393/781] 10.222.0.58 () {56 vars in 1251 bytes} [Tue Jul 26 13:33:58 2022] GET /identity/v3/endpoints?service_id=fcf30d1c2e004d25ad5a47df9d25357d&interface=public®ion_id=RegionOne => generated 194 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:58.495710 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3809baa9-c0ef-4b08-a97f-e22424e5c7aa None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:58.567275 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00ebb12-feee-4abc-a3aa-bf870eb10315 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:58.606731 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75fa1c1-2837-444f-a3c3-1002cb14177d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:58.607389 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75fa1c1-2837-444f-a3c3-1002cb14177d None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:58.607719 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75fa1c1-2837-444f-a3c3-1002cb14177d None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:58.607997 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75fa1c1-2837-444f-a3c3-1002cb14177d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:58.623106 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75fa1c1-2837-444f-a3c3-1002cb14177d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:33:58.628483 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-f75fa1c1-2837-444f-a3c3-1002cb14177d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 13:33:58.628483 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:58.628483 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:58.628483 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:58.628483 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:58.628483 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:58.628483 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:58.628483 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:58.628483 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:58.628483 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:58.628483 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:58.628483 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:58.628483 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:33:58.628483 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:33:58.628483 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:33:58.628483 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:33:58.629563 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:33:58.629563 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:33:58.629563 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:33:58.629563 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:33:58.629563 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:58.629563 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:58.629563 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:58.629563 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:58.629563 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:58.629563 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:58.629563 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:58.629563 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:58.629563 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:58.629563 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:58.629563 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:58.629563 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:58.630423 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:58.630423 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:58.630423 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:58.630423 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:58.630423 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:33:58.630423 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:33:58.630423 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:33:58.630423 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:33:58.630423 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:33:58.630423 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:33:58.630423 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 13:33:58.630423 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:33:58.630423 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 389/782] 10.222.0.58 () {56 vars in 1096 bytes} [Tue Jul 26 13:33:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 13:33:58.637603 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75fa1c1-2837-444f-a3c3-1002cb14177d None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:58.649973 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:58.657795 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3809baa9-c0ef-4b08-a97f-e22424e5c7aa None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:58.699066 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1344487-b8de-4430-92c6-f7ccc621109d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:58.700099 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1344487-b8de-4430-92c6-f7ccc621109d None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:58.700587 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1344487-b8de-4430-92c6-f7ccc621109d None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:58.701097 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1344487-b8de-4430-92c6-f7ccc621109d None admin] PATH_INFO: `/v3/services` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:58.702045 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1344487-b8de-4430-92c6-f7ccc621109d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:58.703670 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1344487-b8de-4430-92c6-f7ccc621109d None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:58.710033 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 394/783] 10.222.0.58 () {56 vars in 1087 bytes} [Tue Jul 26 13:33:58 2022] GET /identity/v3/services => generated 772 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:33:58.746790 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:58.808753 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75fa1c1-2837-444f-a3c3-1002cb14177d None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:58.866327 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22250d2f-0e4c-4e63-beb6-6edf9f29cb0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:58.866987 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22250d2f-0e4c-4e63-beb6-6edf9f29cb0f None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:58.867278 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22250d2f-0e4c-4e63-beb6-6edf9f29cb0f None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:58.868132 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22250d2f-0e4c-4e63-beb6-6edf9f29cb0f None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:58.868897 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22250d2f-0e4c-4e63-beb6-6edf9f29cb0f None admin] RBAC: Adding query filter params (name=admin) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:58.869211 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22250d2f-0e4c-4e63-beb6-6edf9f29cb0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:58.870676 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22250d2f-0e4c-4e63-beb6-6edf9f29cb0f None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:58.885569 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 390/784] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Jul 26 13:33:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:33:58.890859 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1344487-b8de-4430-92c6-f7ccc621109d None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:59.026506 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1344487-b8de-4430-92c6-f7ccc621109d None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:59.073148 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238343c1-545b-41fa-bc6c-2d877f1ded48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:59.073873 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238343c1-545b-41fa-bc6c-2d877f1ded48 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:59.074159 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238343c1-545b-41fa-bc6c-2d877f1ded48 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:59.074436 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238343c1-545b-41fa-bc6c-2d877f1ded48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:59.087507 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238343c1-545b-41fa-bc6c-2d877f1ded48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:59.089037 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238343c1-545b-41fa-bc6c-2d877f1ded48 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:59.107168 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-238343c1-545b-41fa-bc6c-2d877f1ded48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:59.107168 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:59.107168 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:59.107168 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:59.107168 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:59.107168 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:59.107168 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:59.107168 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:59.107168 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:59.107168 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:59.107168 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:59.107168 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:59.107168 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 13:33:59.107168 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 13:33:59.107168 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 13:33:59.108124 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 13:33:59.108124 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:59.108124 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:59.108124 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:59.108124 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:59.108124 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:59.108124 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:59.108124 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:59.108124 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:59.108124 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:59.108124 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:59.108124 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:59.108124 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:59.108124 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:59.108124 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:59.108124 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:59.108124 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:59.109436 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 13:33:59.109436 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 13:33:59.109436 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:59.109436 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:33:59.109436 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 395/785] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:33:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:59.114320 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22250d2f-0e4c-4e63-beb6-6edf9f29cb0f None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:59.221994 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22250d2f-0e4c-4e63-beb6-6edf9f29cb0f None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:59.246065 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:59.268346 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542a4768-daf1-4976-a278-7918717997ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:59.268999 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542a4768-daf1-4976-a278-7918717997ff None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:59.269319 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542a4768-daf1-4976-a278-7918717997ff None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:59.269623 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542a4768-daf1-4976-a278-7918717997ff None admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:59.270335 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542a4768-daf1-4976-a278-7918717997ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:59.270644 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542a4768-daf1-4976-a278-7918717997ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:59.272043 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542a4768-daf1-4976-a278-7918717997ff None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:59.292382 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 391/786] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Jul 26 13:33:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:59.297874 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238343c1-545b-41fa-bc6c-2d877f1ded48 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:59.317676 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:59.402322 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238343c1-545b-41fa-bc6c-2d877f1ded48 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:59.447253 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a045d689-cf14-46e5-bc27-97e37f00e796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:59.447962 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a045d689-cf14-46e5-bc27-97e37f00e796 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:59.448252 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a045d689-cf14-46e5-bc27-97e37f00e796 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:59.448532 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a045d689-cf14-46e5-bc27-97e37f00e796 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:59.456693 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a045d689-cf14-46e5-bc27-97e37f00e796 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:59.458219 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a045d689-cf14-46e5-bc27-97e37f00e796 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:59.474776 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-a045d689-cf14-46e5-bc27-97e37f00e796 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 26 13:33:59.474776 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:59.474776 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:59.474776 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:59.474776 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:59.474776 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:59.474776 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:59.474776 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:59.474776 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:59.474776 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:59.474776 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:59.474776 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:59.474776 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 13:33:59.474776 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 13:33:59.474776 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 13:33:59.474776 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 13:33:59.476199 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:59.476199 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:59.476199 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 13:33:59.476199 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 13:33:59.476199 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 13:33:59.476199 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 13:33:59.476199 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:59.476199 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:59.476199 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:59.476199 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:59.476199 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:59.476199 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:59.476199 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:59.476199 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:59.476199 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:59.476199 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:59.476199 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:59.476199 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:59.477720 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:59.477720 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:59.477720 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 13:33:59.477720 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 13:33:59.477720 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 13:33:59.477720 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 13:33:59.477720 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 13:33:59.477720 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 13:33:59.477720 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 26 13:33:59.477720 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:33:59.480057 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 396/787] 10.222.0.58 () {56 vars in 1096 bytes} [Tue Jul 26 13:33:59 2022] GET /identity/v3/users/swift => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:59.490587 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542a4768-daf1-4976-a278-7918717997ff None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:59.512378 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4890ad-57f6-4e41-a35d-dd9a73613112 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:59.512985 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4890ad-57f6-4e41-a35d-dd9a73613112 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:59.513490 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4890ad-57f6-4e41-a35d-dd9a73613112 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:59.515383 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 397/788] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:33:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:33:59.520925 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7769f8cd-8e97-439f-881d-b9293385c04d None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:59.521465 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7769f8cd-8e97-439f-881d-b9293385c04d None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:59.521971 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7769f8cd-8e97-439f-881d-b9293385c04d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:59.541556 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-7769f8cd-8e97-439f-881d-b9293385c04d None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:33:59.543240 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-7769f8cd-8e97-439f-881d-b9293385c04d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:33:59.625451 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 398/789] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:33:59 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 13:33:59.625451 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180858fa-b1e1-4d9d-8525-69a129cde1e4 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:59.625451 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180858fa-b1e1-4d9d-8525-69a129cde1e4 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:59.625451 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180858fa-b1e1-4d9d-8525-69a129cde1e4 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:59.625451 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 399/790] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:33:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:33:59.631066 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b520e27a-f777-4470-8f84-9de768451d8f None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:59.631352 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b520e27a-f777-4470-8f84-9de768451d8f None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:59.631625 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b520e27a-f777-4470-8f84-9de768451d8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:59.654794 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:59.661674 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-b520e27a-f777-4470-8f84-9de768451d8f None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:33:59.662878 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-b520e27a-f777-4470-8f84-9de768451d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:33:59.667763 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542a4768-daf1-4976-a278-7918717997ff None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:59.706213 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7fa221-6dac-46e9-8444-917fa0877dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:59.706893 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7fa221-6dac-46e9-8444-917fa0877dda None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:59.707176 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7fa221-6dac-46e9-8444-917fa0877dda None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:59.707453 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7fa221-6dac-46e9-8444-917fa0877dda None admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:59.708370 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7fa221-6dac-46e9-8444-917fa0877dda None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:33:59.708681 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7fa221-6dac-46e9-8444-917fa0877dda None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:59.710207 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7fa221-6dac-46e9-8444-917fa0877dda None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:59.733878 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 392/791] 10.222.0.58 () {56 vars in 1135 bytes} [Tue Jul 26 13:33:59 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:33:59.739244 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7fa221-6dac-46e9-8444-917fa0877dda None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:59.740951 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 400/792] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:33:59 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 13:33:59.742180 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b520e27a-f777-4470-8f84-9de768451d8f None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:59.784124 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:59.848153 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7fa221-6dac-46e9-8444-917fa0877dda None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:59.858756 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b520e27a-f777-4470-8f84-9de768451d8f None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:33:59.883778 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e44e0e3-6e32-4562-bd25-1adf9dfcfcc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:59.884428 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e44e0e3-6e32-4562-bd25-1adf9dfcfcc7 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:59.884709 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e44e0e3-6e32-4562-bd25-1adf9dfcfcc7 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:59.884985 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e44e0e3-6e32-4562-bd25-1adf9dfcfcc7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:59.895631 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e44e0e3-6e32-4562-bd25-1adf9dfcfcc7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:33:59.898561 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932dcd61-e418-4d64-bfdf-0cf5e65fa941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:33:59.899890 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932dcd61-e418-4d64-bfdf-0cf5e65fa941 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:33:59.900367 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932dcd61-e418-4d64-bfdf-0cf5e65fa941 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:33:59.900781 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932dcd61-e418-4d64-bfdf-0cf5e65fa941 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:33:59.901067 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-1e44e0e3-6e32-4562-bd25-1adf9dfcfcc7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 13:33:59.901067 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:59.901067 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:59.901067 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:59.901067 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:59.901067 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:59.901067 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:59.901067 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:59.901067 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:59.901067 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:59.901067 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:59.901067 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:59.901067 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:33:59.901067 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:33:59.901067 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:33:59.901067 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:33:59.902195 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:33:59.902195 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:33:59.902195 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:33:59.902195 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:33:59.902195 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:59.902195 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:59.902195 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:59.902195 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:59.902195 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:59.902195 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:59.902195 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:59.902195 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:59.902195 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:59.902195 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:59.902195 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:59.902195 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:59.903200 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:59.903200 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:59.903200 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:59.903200 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:59.903200 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:33:59.903200 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:33:59.903200 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:33:59.903200 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:33:59.903200 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:33:59.903200 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:33:59.903200 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 13:33:59.903200 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:33:59.903200 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 393/793] 10.222.0.58 () {56 vars in 1096 bytes} [Tue Jul 26 13:33:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:59.903902 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e44e0e3-6e32-4562-bd25-1adf9dfcfcc7 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:33:59.914747 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:33:59.925206 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932dcd61-e418-4d64-bfdf-0cf5e65fa941 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:33:59.926689 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932dcd61-e418-4d64-bfdf-0cf5e65fa941 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:33:59.943595 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-932dcd61-e418-4d64-bfdf-0cf5e65fa941 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:59.943595 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:33:59.943595 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:33:59.943595 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:33:59.943595 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:33:59.943595 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:33:59.943595 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:33:59.943595 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:33:59.943595 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:33:59.943595 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:33:59.943595 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:33:59.943595 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:33:59.943595 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 13:33:59.943595 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 13:33:59.943595 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 13:33:59.944489 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 13:33:59.944489 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:33:59.944489 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:33:59.944489 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:33:59.944489 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:33:59.944489 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:33:59.944489 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:33:59.944489 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:33:59.944489 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:33:59.944489 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:33:59.944489 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:33:59.944489 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:33:59.944489 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:33:59.944489 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:33:59.944489 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:33:59.944489 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:33:59.944489 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:33:59.945413 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 13:33:59.945413 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 13:33:59.945413 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:33:59.945413 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:33:59.945413 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 401/794] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:33:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:33:59.946796 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932dcd61-e418-4d64-bfdf-0cf5e65fa941 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:00.015141 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:00.017925 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e44e0e3-6e32-4562-bd25-1adf9dfcfcc7 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:00.056150 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5780f0e-a79b-4c4a-b79e-0d3913b7524b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:00.056895 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5780f0e-a79b-4c4a-b79e-0d3913b7524b None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:00.057272 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5780f0e-a79b-4c4a-b79e-0d3913b7524b None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:00.057688 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5780f0e-a79b-4c4a-b79e-0d3913b7524b None admin] PATH_INFO: `/v3/roles/service` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:00.059211 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932dcd61-e418-4d64-bfdf-0cf5e65fa941 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:00.075982 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5780f0e-a79b-4c4a-b79e-0d3913b7524b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:34:00.082452 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-e5780f0e-a79b-4c4a-b79e-0d3913b7524b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 13:34:00.082452 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:00.082452 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:34:00.082452 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:34:00.082452 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:34:00.082452 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:34:00.082452 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:34:00.082452 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:34:00.082452 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:34:00.082452 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:34:00.082452 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:34:00.082452 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:34:00.082452 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:34:00.082452 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:34:00.082452 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:34:00.082452 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:34:00.083583 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:34:00.083583 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:34:00.083583 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:34:00.083583 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:34:00.083583 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:34:00.083583 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:34:00.083583 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:34:00.083583 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:34:00.083583 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:34:00.083583 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:34:00.083583 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:34:00.083583 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:34:00.083583 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:34:00.083583 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:34:00.083583 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:34:00.083583 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:34:00.084409 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:34:00.084409 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:34:00.084409 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:34:00.084409 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:34:00.084409 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:34:00.084409 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:34:00.084409 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:34:00.084409 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:34:00.084409 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:34:00.084409 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:34:00.084409 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 13:34:00.084409 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:34:00.089591 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 394/795] 10.222.0.58 () {56 vars in 1102 bytes} [Tue Jul 26 13:33:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:34:00.090993 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5780f0e-a79b-4c4a-b79e-0d3913b7524b None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:00.100824 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4da846e-e7a0-4020-afa1-69ba3d6a6606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:00.101919 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4da846e-e7a0-4020-afa1-69ba3d6a6606 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:00.102382 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4da846e-e7a0-4020-afa1-69ba3d6a6606 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:00.102833 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4da846e-e7a0-4020-afa1-69ba3d6a6606 None admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:00.103913 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4da846e-e7a0-4020-afa1-69ba3d6a6606 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:00.104402 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4da846e-e7a0-4020-afa1-69ba3d6a6606 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:00.106331 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4da846e-e7a0-4020-afa1-69ba3d6a6606 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:00.121443 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 402/796] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Jul 26 13:33:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:00.122942 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4da846e-e7a0-4020-afa1-69ba3d6a6606 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:00.179061 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:00.202843 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5780f0e-a79b-4c4a-b79e-0d3913b7524b None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:00.236017 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4da846e-e7a0-4020-afa1-69ba3d6a6606 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:00.241013 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8412a0-4763-41f7-a714-71eee80259d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:00.242033 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8412a0-4763-41f7-a714-71eee80259d9 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:00.243100 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8412a0-4763-41f7-a714-71eee80259d9 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:00.243553 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8412a0-4763-41f7-a714-71eee80259d9 None admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:00.244460 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8412a0-4763-41f7-a714-71eee80259d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:00.244983 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8412a0-4763-41f7-a714-71eee80259d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:00.246721 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8412a0-4763-41f7-a714-71eee80259d9 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:00.260667 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 395/797] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Jul 26 13:34:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:00.261889 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8412a0-4763-41f7-a714-71eee80259d9 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:00.276421 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605a8c8c-a632-4e58-b1da-068b25224761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:00.277195 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a8c8c-a632-4e58-b1da-068b25224761 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:00.277601 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a8c8c-a632-4e58-b1da-068b25224761 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:00.277949 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a8c8c-a632-4e58-b1da-068b25224761 None admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:00.278744 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605a8c8c-a632-4e58-b1da-068b25224761 None admin] RBAC: Adding query filter params (name=service) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:00.279238 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605a8c8c-a632-4e58-b1da-068b25224761 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:00.280795 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605a8c8c-a632-4e58-b1da-068b25224761 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:00.287310 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:00.296359 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 403/798] 10.222.0.58 () {56 vars in 1103 bytes} [Tue Jul 26 13:34:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:00.296359 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605a8c8c-a632-4e58-b1da-068b25224761 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:00.398376 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8412a0-4763-41f7-a714-71eee80259d9 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:00.414918 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605a8c8c-a632-4e58-b1da-068b25224761 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:00.433434 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:00.438175 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60296423-ee5d-429b-9497-fe0b98d6be75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:00.439965 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60296423-ee5d-429b-9497-fe0b98d6be75 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:00.440417 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60296423-ee5d-429b-9497-fe0b98d6be75 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:00.440837 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60296423-ee5d-429b-9497-fe0b98d6be75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:00.456730 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60296423-ee5d-429b-9497-fe0b98d6be75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:00.457919 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abba0ccc-2e8b-447d-9feb-a608a0c532dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:00.458710 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abba0ccc-2e8b-447d-9feb-a608a0c532dc None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:00.459000 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60296423-ee5d-429b-9497-fe0b98d6be75 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:00.459170 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abba0ccc-2e8b-447d-9feb-a608a0c532dc None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:00.459539 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abba0ccc-2e8b-447d-9feb-a608a0c532dc None admin] PATH_INFO: `/v3/projects/service` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:00.466800 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abba0ccc-2e8b-447d-9feb-a608a0c532dc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:00.468392 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abba0ccc-2e8b-447d-9feb-a608a0c532dc None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:00.477885 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-60296423-ee5d-429b-9497-fe0b98d6be75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:34:00.477885 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:00.477885 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:34:00.477885 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:34:00.477885 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:34:00.477885 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:34:00.477885 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:34:00.477885 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:34:00.477885 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:34:00.477885 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:34:00.477885 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:34:00.477885 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:34:00.477885 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 13:34:00.477885 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 13:34:00.477885 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 13:34:00.478948 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 13:34:00.478948 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:34:00.478948 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:34:00.478948 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:34:00.478948 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:34:00.478948 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:34:00.478948 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:34:00.478948 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:34:00.478948 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:34:00.478948 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:34:00.478948 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:34:00.478948 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:34:00.478948 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:34:00.478948 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:34:00.478948 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:34:00.478948 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:34:00.478948 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:34:00.480154 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 13:34:00.480154 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 13:34:00.480154 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:34:00.480154 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:34:00.486490 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-abba0ccc-2e8b-447d-9feb-a608a0c532dc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 13:34:00.486490 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:00.486490 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:34:00.486490 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:34:00.486490 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:34:00.486490 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:34:00.486490 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:34:00.486490 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:34:00.486490 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:34:00.486490 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:34:00.486490 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:34:00.486490 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:34:00.486490 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 13:34:00.486490 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 13:34:00.486490 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 13:34:00.487934 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 13:34:00.487934 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:34:00.487934 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:34:00.487934 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:34:00.487934 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:34:00.487934 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:34:00.487934 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:34:00.487934 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:34:00.487934 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:34:00.487934 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:34:00.487934 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:34:00.487934 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:34:00.487934 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:34:00.487934 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:34:00.487934 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:34:00.487934 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:34:00.487934 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:34:00.489219 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 13:34:00.489219 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 13:34:00.489219 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 13:34:00.489219 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 13:34:00.489219 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 13:34:00.489219 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 13:34:00.489219 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 13:34:00.489219 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:34:00.489219 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 404/800] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Jul 26 13:34:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:34:00.489219 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abba0ccc-2e8b-447d-9feb-a608a0c532dc None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:00.489996 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 396/800] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:34:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:34:00.492648 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60296423-ee5d-429b-9497-fe0b98d6be75 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:00.579185 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:00.612706 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60296423-ee5d-429b-9497-fe0b98d6be75 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:00.627219 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abba0ccc-2e8b-447d-9feb-a608a0c532dc None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:00.651564 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ad1c5d-eac2-4cbf-aa2d-a698a4681088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:00.652307 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ad1c5d-eac2-4cbf-aa2d-a698a4681088 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:00.652668 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ad1c5d-eac2-4cbf-aa2d-a698a4681088 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:00.653066 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ad1c5d-eac2-4cbf-aa2d-a698a4681088 None admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:00.654011 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ad1c5d-eac2-4cbf-aa2d-a698a4681088 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:00.654429 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ad1c5d-eac2-4cbf-aa2d-a698a4681088 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:00.656125 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ad1c5d-eac2-4cbf-aa2d-a698a4681088 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:00.669447 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 397/801] 10.222.0.58 () {56 vars in 1148 bytes} [Tue Jul 26 13:34:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:00.670445 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ad1c5d-eac2-4cbf-aa2d-a698a4681088 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:00.683307 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fee3810-e233-4827-b1ba-a9dbb8749ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:00.684095 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee3810-e233-4827-b1ba-a9dbb8749ede None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:00.684756 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee3810-e233-4827-b1ba-a9dbb8749ede None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:00.685139 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee3810-e233-4827-b1ba-a9dbb8749ede None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:00.698499 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fee3810-e233-4827-b1ba-a9dbb8749ede None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:00.699973 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fee3810-e233-4827-b1ba-a9dbb8749ede None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:00.717875 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-4fee3810-e233-4827-b1ba-a9dbb8749ede None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:34:00.717875 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:00.717875 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:34:00.717875 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:34:00.717875 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:34:00.717875 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:34:00.717875 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:34:00.717875 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:34:00.717875 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:34:00.717875 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:34:00.717875 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:34:00.717875 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:34:00.717875 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 13:34:00.717875 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 13:34:00.717875 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 13:34:00.719364 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 13:34:00.719364 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:34:00.719364 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:34:00.719364 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:34:00.719364 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:34:00.719364 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:34:00.719364 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:34:00.719364 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:34:00.719364 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:34:00.719364 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:34:00.719364 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:34:00.719364 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:34:00.719364 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:34:00.719364 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:34:00.719364 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:34:00.719364 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:34:00.719364 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:34:00.720865 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 13:34:00.720865 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 13:34:00.720865 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:34:00.720865 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:34:00.724258 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 405/802] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:34:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:34:00.725148 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fee3810-e233-4827-b1ba-a9dbb8749ede None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:00.758930 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:00.792189 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ad1c5d-eac2-4cbf-aa2d-a698a4681088 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:00.828886 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:00.836721 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be36d518-fa8b-4d82-8a01-deff4c41c845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:00.837614 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be36d518-fa8b-4d82-8a01-deff4c41c845 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:00.837984 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be36d518-fa8b-4d82-8a01-deff4c41c845 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:00.838333 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be36d518-fa8b-4d82-8a01-deff4c41c845 None admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:00.839131 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be36d518-fa8b-4d82-8a01-deff4c41c845 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:00.839520 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be36d518-fa8b-4d82-8a01-deff4c41c845 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:00.841015 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be36d518-fa8b-4d82-8a01-deff4c41c845 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:00.856655 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fee3810-e233-4827-b1ba-a9dbb8749ede None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:00.860533 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 398/803] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Jul 26 13:34:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:00.862044 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2261db-92c7-4de6-a416-28389b4c5fab None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:00.862338 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2261db-92c7-4de6-a416-28389b4c5fab None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:00.862618 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2261db-92c7-4de6-a416-28389b4c5fab None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:00.864149 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 399/804] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:34:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:34:00.864797 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2261db-92c7-4de6-a416-28389b4c5fab None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:00.899318 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b976134-326e-430a-b4fe-0b9c3fe8af7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:00.900203 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b976134-326e-430a-b4fe-0b9c3fe8af7f None admin] REQUEST_METHOD: `PUT` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:00.900588 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b976134-326e-430a-b4fe-0b9c3fe8af7f None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:00.900884 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b976134-326e-430a-b4fe-0b9c3fe8af7f None admin] PATH_INFO: `/v3/projects/1e8bb4385c204210a1bc77b1f0d654f3/users/3ab2819bac844163ae4305e4f2186605/roles/6e0201a9a6484f7d8b112b1eafff70bd` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:00.903128 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b976134-326e-430a-b4fe-0b9c3fe8af7f None admin] RBAC: Authorizing `identity:create_grant(project_id=1e8bb4385c204210a1bc77b1f0d654f3, user_id=3ab2819bac844163ae4305e4f2186605, role_id=6e0201a9a6484f7d8b112b1eafff70bd)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:00.904535 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b976134-326e-430a-b4fe-0b9c3fe8af7f None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:00.921524 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 406/805] 10.222.0.58 () {58 vars in 1439 bytes} [Tue Jul 26 13:34:00 2022] PUT /identity/v3/projects/1e8bb4385c204210a1bc77b1f0d654f3/users/3ab2819bac844163ae4305e4f2186605/roles/6e0201a9a6484f7d8b112b1eafff70bd => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:34:00.921909 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b976134-326e-430a-b4fe-0b9c3fe8af7f None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:01.016308 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2261db-92c7-4de6-a416-28389b4c5fab None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:01.054818 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:01.079134 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b976134-326e-430a-b4fe-0b9c3fe8af7f None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:01.083124 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3430d8-a350-411d-89fd-2925d6ed8d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:01.083751 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3430d8-a350-411d-89fd-2925d6ed8d3a None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:01.084157 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3430d8-a350-411d-89fd-2925d6ed8d3a None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:01.084531 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3430d8-a350-411d-89fd-2925d6ed8d3a None admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:01.085342 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3430d8-a350-411d-89fd-2925d6ed8d3a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:01.085788 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3430d8-a350-411d-89fd-2925d6ed8d3a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:01.087900 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3430d8-a350-411d-89fd-2925d6ed8d3a None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:01.102220 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 400/806] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Jul 26 13:34:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:01.103634 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673bb26b-740c-495f-80a6-eaeb71033e40 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:01.104022 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673bb26b-740c-495f-80a6-eaeb71033e40 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:01.104394 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673bb26b-740c-495f-80a6-eaeb71033e40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:01.121239 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad1d55c-6093-41a9-8a09-422d15464a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:01.122637 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad1d55c-6093-41a9-8a09-422d15464a87 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:01.122921 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad1d55c-6093-41a9-8a09-422d15464a87 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:01.123191 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad1d55c-6093-41a9-8a09-422d15464a87 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:01.127685 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-673bb26b-740c-495f-80a6-eaeb71033e40 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:34:01.128884 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-673bb26b-740c-495f-80a6-eaeb71033e40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:34:01.133320 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad1d55c-6093-41a9-8a09-422d15464a87 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:01.134816 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad1d55c-6093-41a9-8a09-422d15464a87 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:01.149704 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-bad1d55c-6093-41a9-8a09-422d15464a87 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 26 13:34:01.149704 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:01.149704 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:34:01.149704 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:34:01.149704 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:34:01.149704 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:34:01.149704 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:34:01.149704 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:34:01.149704 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:34:01.149704 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:34:01.149704 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:34:01.149704 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:34:01.149704 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 13:34:01.149704 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 13:34:01.149704 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 13:34:01.149704 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 13:34:01.150724 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:34:01.150724 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:34:01.150724 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 13:34:01.150724 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 13:34:01.150724 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 13:34:01.150724 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 13:34:01.150724 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:34:01.150724 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:34:01.150724 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:34:01.150724 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:34:01.150724 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:34:01.150724 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:34:01.150724 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:34:01.150724 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:34:01.150724 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:34:01.150724 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:34:01.150724 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:34:01.150724 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:34:01.151697 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:34:01.151697 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:34:01.151697 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 13:34:01.151697 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 13:34:01.151697 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 13:34:01.151697 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 13:34:01.151697 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 13:34:01.151697 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 13:34:01.151697 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 26 13:34:01.151697 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:34:01.151697 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 407/807] 10.222.0.58 () {56 vars in 1093 bytes} [Tue Jul 26 13:34:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:34:01.152279 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad1d55c-6093-41a9-8a09-422d15464a87 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:01.196930 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:01.220654 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 401/808] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:34:01 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 13:34:01.221949 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673bb26b-740c-495f-80a6-eaeb71033e40 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:01.275730 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad1d55c-6093-41a9-8a09-422d15464a87 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:01.312600 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6884a16-5681-426a-8030-0657ec050552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:01.313308 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6884a16-5681-426a-8030-0657ec050552 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:01.313750 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6884a16-5681-426a-8030-0657ec050552 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:01.314030 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6884a16-5681-426a-8030-0657ec050552 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:01.325264 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6884a16-5681-426a-8030-0657ec050552 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:01.327133 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6884a16-5681-426a-8030-0657ec050552 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:01.340076 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-e6884a16-5681-426a-8030-0657ec050552 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 13:34:01.340076 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:01.340076 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:34:01.340076 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:34:01.340076 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:34:01.340076 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:34:01.340076 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:34:01.340076 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:34:01.340076 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:34:01.340076 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:34:01.340076 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:34:01.340076 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:34:01.340076 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 13:34:01.340076 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 13:34:01.340076 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 13:34:01.340076 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 13:34:01.341224 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:34:01.341224 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:34:01.341224 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 13:34:01.341224 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 13:34:01.341224 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 13:34:01.341224 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 13:34:01.341224 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:34:01.341224 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:34:01.341224 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:34:01.341224 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:34:01.341224 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:34:01.341224 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:34:01.341224 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:34:01.341224 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:34:01.341224 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:34:01.341224 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:34:01.341224 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:34:01.341224 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:34:01.342297 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:34:01.342297 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:34:01.342297 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 13:34:01.342297 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 13:34:01.342297 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 13:34:01.342297 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 13:34:01.342297 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 13:34:01.342297 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 13:34:01.342297 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 13:34:01.342297 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:34:01.342297 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 408/809] 10.222.0.58 () {56 vars in 1117 bytes} [Tue Jul 26 13:34:01 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:34:01.342872 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5d4c7e-5e36-4292-9af5-8a040f7065d4 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:01.343127 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5d4c7e-5e36-4292-9af5-8a040f7065d4 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:01.343248 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5d4c7e-5e36-4292-9af5-8a040f7065d4 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:01.347628 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673bb26b-740c-495f-80a6-eaeb71033e40 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:01.348481 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 409/810] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:34:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:34:01.349222 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5d4c7e-5e36-4292-9af5-8a040f7065d4 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:01.353008 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:01.382139 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff8dbb0-3d0d-4506-b096-63827aeca720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:01.383332 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff8dbb0-3d0d-4506-b096-63827aeca720 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:01.383628 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff8dbb0-3d0d-4506-b096-63827aeca720 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:01.383905 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff8dbb0-3d0d-4506-b096-63827aeca720 None admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:01.384706 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff8dbb0-3d0d-4506-b096-63827aeca720 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:01.385015 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff8dbb0-3d0d-4506-b096-63827aeca720 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:01.386502 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff8dbb0-3d0d-4506-b096-63827aeca720 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:01.410542 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 402/811] 10.222.0.58 () {56 vars in 1133 bytes} [Tue Jul 26 13:34:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:01.410542 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff8dbb0-3d0d-4506-b096-63827aeca720 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:01.452389 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:01.473467 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5d4c7e-5e36-4292-9af5-8a040f7065d4 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:01.514454 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb478473-f0e7-4578-b349-65ff5fb2e05b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:01.515088 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb478473-f0e7-4578-b349-65ff5fb2e05b None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:01.515374 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb478473-f0e7-4578-b349-65ff5fb2e05b None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:01.515652 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb478473-f0e7-4578-b349-65ff5fb2e05b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:01.515992 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff8dbb0-3d0d-4506-b096-63827aeca720 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:01.516401 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb478473-f0e7-4578-b349-65ff5fb2e05b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:01.517777 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb478473-f0e7-4578-b349-65ff5fb2e05b None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:01.526977 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-cb478473-f0e7-4578-b349-65ff5fb2e05b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 26 13:34:01.526977 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:01.526977 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 13:34:01.526977 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 13:34:01.526977 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 13:34:01.526977 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 13:34:01.526977 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 13:34:01.526977 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 13:34:01.526977 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 26 13:34:01.526977 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:34:01.526977 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 13:34:01.526977 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:34:01.526977 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:01.526977 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:34:01.526977 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:34:01.526977 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:34:01.527921 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:34:01.527921 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:34:01.527921 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:34:01.527921 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:34:01.527921 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:34:01.527921 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:34:01.527921 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:34:01.527921 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 13:34:01.527921 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 13:34:01.527921 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 13:34:01.527921 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 13:34:01.527921 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:34:01.527921 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:34:01.527921 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:34:01.527921 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:34:01.527921 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:34:01.528764 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:34:01.528764 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:34:01.528764 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:34:01.528764 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:34:01.528764 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:34:01.528764 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:34:01.528764 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:34:01.528764 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:34:01.528764 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:34:01.528764 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:34:01.528764 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:34:01.528764 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 13:34:01.528764 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 13:34:01.528764 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 26 13:34:01.528764 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:34:01.528764 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 410/812] 10.222.0.58 () {56 vars in 1129 bytes} [Tue Jul 26 13:34:01 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:34:01.529631 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194d6487-310e-4ab7-b07b-be855b34b94f None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:01.529631 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194d6487-310e-4ab7-b07b-be855b34b94f None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:01.530579 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194d6487-310e-4ab7-b07b-be855b34b94f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:01.548888 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-194d6487-310e-4ab7-b07b-be855b34b94f None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:34:01.550116 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-194d6487-310e-4ab7-b07b-be855b34b94f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:34:01.557673 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdd2c52-b8d0-44cc-bdf9-908229eb15ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:01.558284 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdd2c52-b8d0-44cc-bdf9-908229eb15ba None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:01.558569 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdd2c52-b8d0-44cc-bdf9-908229eb15ba None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:01.558845 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdd2c52-b8d0-44cc-bdf9-908229eb15ba None admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:01.559636 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdd2c52-b8d0-44cc-bdf9-908229eb15ba None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:01.559949 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdd2c52-b8d0-44cc-bdf9-908229eb15ba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:01.561358 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdd2c52-b8d0-44cc-bdf9-908229eb15ba None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:01.578959 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 403/813] 10.222.0.58 () {56 vars in 1149 bytes} [Tue Jul 26 13:34:01 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:01.579781 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdd2c52-b8d0-44cc-bdf9-908229eb15ba None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:01.621652 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:01.634008 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 411/814] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:34:01 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 13:34:01.635435 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194d6487-310e-4ab7-b07b-be855b34b94f None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:01.687897 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:01.693386 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdd2c52-b8d0-44cc-bdf9-908229eb15ba None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:01.731256 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6b7b98-4813-4ab2-a28e-ce2fc949f621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:01.731809 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6b7b98-4813-4ab2-a28e-ce2fc949f621 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:01.732089 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6b7b98-4813-4ab2-a28e-ce2fc949f621 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:01.732362 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6b7b98-4813-4ab2-a28e-ce2fc949f621 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:01.745534 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194d6487-310e-4ab7-b07b-be855b34b94f None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:01.781595 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a1d290-7a24-4e5a-a521-1d511dad0b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:01.782220 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a1d290-7a24-4e5a-a521-1d511dad0b41 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:01.782507 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a1d290-7a24-4e5a-a521-1d511dad0b41 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:01.782782 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a1d290-7a24-4e5a-a521-1d511dad0b41 None admin] PATH_INFO: `/v3/services` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:01.783459 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a1d290-7a24-4e5a-a521-1d511dad0b41 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:01.783765 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a1d290-7a24-4e5a-a521-1d511dad0b41 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:01.785125 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a1d290-7a24-4e5a-a521-1d511dad0b41 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:01.793512 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 412/815] 10.222.0.58 () {56 vars in 1124 bytes} [Tue Jul 26 13:34:01 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:01.793512 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a1d290-7a24-4e5a-a521-1d511dad0b41 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:01.810516 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6b7b98-4813-4ab2-a28e-ce2fc949f621 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:01.812317 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6b7b98-4813-4ab2-a28e-ce2fc949f621 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:01.858798 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:01.903044 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a1d290-7a24-4e5a-a521-1d511dad0b41 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:01.916507 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:01.925952 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 404/816] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Jul 26 13:34:01 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 13:34:01.925952 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6b7b98-4813-4ab2-a28e-ce2fc949f621 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:01.946030 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2728bae8-76e4-4c80-aacb-199b4b4a6bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:01.946720 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2728bae8-76e4-4c80-aacb-199b4b4a6bdb None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:01.947004 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2728bae8-76e4-4c80-aacb-199b4b4a6bdb None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:01.947276 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2728bae8-76e4-4c80-aacb-199b4b4a6bdb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:01.957600 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2728bae8-76e4-4c80-aacb-199b4b4a6bdb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:01.959020 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2728bae8-76e4-4c80-aacb-199b4b4a6bdb None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:01.975346 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-2728bae8-76e4-4c80-aacb-199b4b4a6bdb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:34:01.975346 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:01.975346 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:34:01.975346 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:34:01.975346 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:34:01.975346 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:34:01.975346 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:34:01.975346 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:34:01.975346 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:34:01.975346 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:34:01.975346 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:34:01.975346 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:34:01.975346 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 13:34:01.975346 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 13:34:01.975346 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 13:34:01.976239 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 13:34:01.976239 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:34:01.976239 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:34:01.976239 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:34:01.976239 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:34:01.976239 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:34:01.976239 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:34:01.976239 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:34:01.976239 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:34:01.976239 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:34:01.976239 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:34:01.976239 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:34:01.976239 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:34:01.976239 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:34:01.976239 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:34:01.976239 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:34:01.976239 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:34:01.977694 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 13:34:01.977694 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 13:34:01.977694 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:34:01.977694 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:34:01.981673 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 413/817] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:34:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:34:01.982456 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2728bae8-76e4-4c80-aacb-199b4b4a6bdb None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:02.034906 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6b7b98-4813-4ab2-a28e-ce2fc949f621 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:02.074814 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae79110-4cdf-4b53-9713-60c9bae6f899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:02.075872 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae79110-4cdf-4b53-9713-60c9bae6f899 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:02.077075 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae79110-4cdf-4b53-9713-60c9bae6f899 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:02.077611 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae79110-4cdf-4b53-9713-60c9bae6f899 None admin] PATH_INFO: `/v3/services/network` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:02.078479 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae79110-4cdf-4b53-9713-60c9bae6f899 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:02.080053 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae79110-4cdf-4b53-9713-60c9bae6f899 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:02.090609 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-1ae79110-4cdf-4b53-9713-60c9bae6f899 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 26 13:34:02.090609 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:02.090609 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 13:34:02.090609 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 13:34:02.090609 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 13:34:02.090609 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 13:34:02.090609 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 13:34:02.090609 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 13:34:02.090609 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 26 13:34:02.090609 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:34:02.090609 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 13:34:02.090609 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:34:02.090609 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:02.090609 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:34:02.090609 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:34:02.090609 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:34:02.090609 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:34:02.092016 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:34:02.092016 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:34:02.092016 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:34:02.092016 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:34:02.092016 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:34:02.092016 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:34:02.092016 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 13:34:02.092016 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 13:34:02.092016 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 13:34:02.092016 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 13:34:02.092016 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:34:02.092016 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:34:02.092016 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:34:02.092016 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:34:02.092016 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:34:02.092016 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:34:02.092904 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:34:02.092904 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:34:02.092904 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:34:02.092904 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:34:02.092904 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:34:02.092904 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:34:02.092904 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:34:02.092904 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:34:02.092904 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:34:02.092904 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:34:02.092904 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 13:34:02.092904 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 13:34:02.092904 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 26 13:34:02.092904 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:34:02.092904 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 405/818] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Jul 26 13:34:01 2022] GET /identity/v3/services/network => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:34:02.093711 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae79110-4cdf-4b53-9713-60c9bae6f899 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:02.099182 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2728bae8-76e4-4c80-aacb-199b4b4a6bdb None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:02.115489 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:02.138896 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65a734d-91e6-4aa5-953d-dd0477b002d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:02.139525 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65a734d-91e6-4aa5-953d-dd0477b002d0 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:02.140031 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65a734d-91e6-4aa5-953d-dd0477b002d0 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:02.140206 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65a734d-91e6-4aa5-953d-dd0477b002d0 None admin] PATH_INFO: `/v3/services` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:02.140878 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65a734d-91e6-4aa5-953d-dd0477b002d0 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:02.141414 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65a734d-91e6-4aa5-953d-dd0477b002d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:02.142698 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65a734d-91e6-4aa5-953d-dd0477b002d0 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:02.153147 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 414/819] 10.222.0.58 () {56 vars in 1124 bytes} [Tue Jul 26 13:34:01 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:02.154154 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65a734d-91e6-4aa5-953d-dd0477b002d0 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:02.159859 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:02.204981 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae79110-4cdf-4b53-9713-60c9bae6f899 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:02.240345 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f0d816-f993-4db6-ab95-9fc9c8369872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:02.240907 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f0d816-f993-4db6-ab95-9fc9c8369872 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:02.241185 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f0d816-f993-4db6-ab95-9fc9c8369872 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:02.241455 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f0d816-f993-4db6-ab95-9fc9c8369872 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:02.252271 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f0d816-f993-4db6-ab95-9fc9c8369872 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:02.253731 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f0d816-f993-4db6-ab95-9fc9c8369872 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:02.259484 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65a734d-91e6-4aa5-953d-dd0477b002d0 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:02.268442 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-c0f0d816-f993-4db6-ab95-9fc9c8369872 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:34:02.268442 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:02.268442 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:34:02.268442 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:34:02.268442 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:34:02.268442 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:34:02.268442 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:34:02.268442 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:34:02.268442 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:34:02.268442 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:34:02.268442 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:34:02.268442 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:34:02.268442 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 13:34:02.268442 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 13:34:02.268442 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 13:34:02.269335 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 13:34:02.269335 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:34:02.269335 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:34:02.269335 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:34:02.269335 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:34:02.269335 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:34:02.269335 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:34:02.269335 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:34:02.269335 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:34:02.269335 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:34:02.269335 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:34:02.269335 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:34:02.269335 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:34:02.269335 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:34:02.269335 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:34:02.269335 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:34:02.269335 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:34:02.270274 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 13:34:02.270274 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 13:34:02.270274 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:34:02.270274 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:34:02.270274 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 406/820] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:34:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:34:02.270562 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f0d816-f993-4db6-ab95-9fc9c8369872 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:02.301278 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a359eda8-128b-4fcc-b07b-a0d2fb6517e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:02.301933 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a359eda8-128b-4fcc-b07b-a0d2fb6517e1 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:02.302214 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a359eda8-128b-4fcc-b07b-a0d2fb6517e1 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:02.302489 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a359eda8-128b-4fcc-b07b-a0d2fb6517e1 None admin] PATH_INFO: `/v3/domains` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:02.303178 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a359eda8-128b-4fcc-b07b-a0d2fb6517e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:02.303486 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a359eda8-128b-4fcc-b07b-a0d2fb6517e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:02.304836 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a359eda8-128b-4fcc-b07b-a0d2fb6517e1 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:02.317255 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 415/821] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Jul 26 13:34:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:02.318073 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a359eda8-128b-4fcc-b07b-a0d2fb6517e1 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:02.366651 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:02.383521 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f0d816-f993-4db6-ab95-9fc9c8369872 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:02.397769 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:02.424639 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b625ac0d-19fc-4078-a13f-8fe894cf5487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:02.425264 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b625ac0d-19fc-4078-a13f-8fe894cf5487 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:02.425579 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b625ac0d-19fc-4078-a13f-8fe894cf5487 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:02.425862 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b625ac0d-19fc-4078-a13f-8fe894cf5487 None admin] PATH_INFO: `/v3/services` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:02.426543 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b625ac0d-19fc-4078-a13f-8fe894cf5487 None admin] RBAC: Adding query filter params (name=network) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:02.426853 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b625ac0d-19fc-4078-a13f-8fe894cf5487 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:02.428311 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b625ac0d-19fc-4078-a13f-8fe894cf5487 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:02.437255 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 407/822] 10.222.0.58 () {56 vars in 1112 bytes} [Tue Jul 26 13:34:02 2022] GET /identity/v3/services?name=network => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:02.437408 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a359eda8-128b-4fcc-b07b-a0d2fb6517e1 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:02.437523 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b625ac0d-19fc-4078-a13f-8fe894cf5487 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:02.475776 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaa10cf-d8db-4d74-8aa8-afa644d1f90b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:02.476569 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaa10cf-d8db-4d74-8aa8-afa644d1f90b None admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:02.477504 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaa10cf-d8db-4d74-8aa8-afa644d1f90b None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:02.477846 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaa10cf-d8db-4d74-8aa8-afa644d1f90b None admin] PATH_INFO: `/v3/endpoints` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:02.478628 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcaa10cf-d8db-4d74-8aa8-afa644d1f90b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:02.479990 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcaa10cf-d8db-4d74-8aa8-afa644d1f90b None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:02.509606 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 416/823] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 13:34:02 2022] POST /identity/v3/endpoints => generated 346 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:34:02.509606 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaa10cf-d8db-4d74-8aa8-afa644d1f90b None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:02.575580 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b625ac0d-19fc-4078-a13f-8fe894cf5487 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:02.628684 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3eaf42-77b1-4118-9d0b-304aeb5ac8ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:02.629782 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3eaf42-77b1-4118-9d0b-304aeb5ac8ab None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:02.630360 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3eaf42-77b1-4118-9d0b-304aeb5ac8ab None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:02.630640 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3eaf42-77b1-4118-9d0b-304aeb5ac8ab None admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:02.631350 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3eaf42-77b1-4118-9d0b-304aeb5ac8ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:02.631558 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:02.631982 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3eaf42-77b1-4118-9d0b-304aeb5ac8ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:02.633965 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3eaf42-77b1-4118-9d0b-304aeb5ac8ab None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:02.643050 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaa10cf-d8db-4d74-8aa8-afa644d1f90b None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:02.652130 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 408/824] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Jul 26 13:34:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:02.653560 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3eaf42-77b1-4118-9d0b-304aeb5ac8ab None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:02.689939 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04e68d0-f5fa-43c1-bd1a-27ed15721ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:02.690641 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04e68d0-f5fa-43c1-bd1a-27ed15721ae0 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:02.691065 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04e68d0-f5fa-43c1-bd1a-27ed15721ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:02.691341 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04e68d0-f5fa-43c1-bd1a-27ed15721ae0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:02.697095 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:02.700663 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04e68d0-f5fa-43c1-bd1a-27ed15721ae0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:02.703093 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04e68d0-f5fa-43c1-bd1a-27ed15721ae0 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:02.717668 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-b04e68d0-f5fa-43c1-bd1a-27ed15721ae0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 13:34:02.717668 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:02.717668 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:34:02.717668 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:34:02.717668 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:34:02.717668 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:34:02.717668 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:34:02.717668 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:34:02.717668 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:34:02.717668 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:34:02.717668 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:34:02.717668 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:34:02.717668 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 13:34:02.717668 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 13:34:02.717668 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 13:34:02.718741 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 13:34:02.718741 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:34:02.718741 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:34:02.718741 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:34:02.718741 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:34:02.718741 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:34:02.718741 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:34:02.718741 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:34:02.718741 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:34:02.718741 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:34:02.718741 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:34:02.718741 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:34:02.718741 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:34:02.718741 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:34:02.718741 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:34:02.718741 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:34:02.718741 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:34:02.719778 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 13:34:02.719778 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 13:34:02.719778 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 13:34:02.719778 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 13:34:02.719778 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 13:34:02.719778 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 13:34:02.719778 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 13:34:02.719778 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:34:02.722032 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 417/825] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Jul 26 13:34:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:34:02.723023 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04e68d0-f5fa-43c1-bd1a-27ed15721ae0 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:02.778516 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3eaf42-77b1-4118-9d0b-304aeb5ac8ab None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:02.817024 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db2eaaf-a752-432f-a808-29e484f19089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:02.817821 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db2eaaf-a752-432f-a808-29e484f19089 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:02.818180 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db2eaaf-a752-432f-a808-29e484f19089 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:02.818530 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db2eaaf-a752-432f-a808-29e484f19089 None admin] PATH_INFO: `/v3/services` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:02.819283 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db2eaaf-a752-432f-a808-29e484f19089 None admin] RBAC: Adding query filter params (type=network) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:02.819661 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db2eaaf-a752-432f-a808-29e484f19089 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:02.821842 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db2eaaf-a752-432f-a808-29e484f19089 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:02.827309 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 409/826] 10.222.0.58 () {56 vars in 1112 bytes} [Tue Jul 26 13:34:02 2022] GET /identity/v3/services?type=network => generated 350 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:02.828254 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db2eaaf-a752-432f-a808-29e484f19089 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:02.846121 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04e68d0-f5fa-43c1-bd1a-27ed15721ae0 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:02.891248 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453a7994-275b-49b0-90fa-f86e02aefc91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:02.892893 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453a7994-275b-49b0-90fa-f86e02aefc91 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:02.893369 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453a7994-275b-49b0-90fa-f86e02aefc91 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:02.893857 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453a7994-275b-49b0-90fa-f86e02aefc91 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:02.905223 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453a7994-275b-49b0-90fa-f86e02aefc91 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:02.907071 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453a7994-275b-49b0-90fa-f86e02aefc91 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:02.923310 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-453a7994-275b-49b0-90fa-f86e02aefc91 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 13:34:02.923310 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:02.923310 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:34:02.923310 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:34:02.923310 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:34:02.923310 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:34:02.923310 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:34:02.923310 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:34:02.923310 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:34:02.923310 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:34:02.923310 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:34:02.923310 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:34:02.923310 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 13:34:02.923310 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 13:34:02.923310 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 13:34:02.924250 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 13:34:02.924250 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:34:02.924250 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:34:02.924250 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:34:02.924250 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:34:02.924250 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:34:02.924250 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:34:02.924250 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:34:02.924250 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:34:02.924250 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:34:02.924250 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:34:02.924250 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:34:02.924250 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:34:02.924250 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:34:02.924250 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:34:02.924250 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:34:02.924250 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:34:02.925137 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 13:34:02.925137 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 13:34:02.925137 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 13:34:02.925137 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 13:34:02.925137 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 13:34:02.925137 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 13:34:02.925137 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 13:34:02.925137 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:34:02.932140 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 418/827] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Jul 26 13:34:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:34:02.932140 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453a7994-275b-49b0-90fa-f86e02aefc91 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:02.940473 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:02.944575 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db2eaaf-a752-432f-a808-29e484f19089 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:02.956675 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:02.982940 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f65daff-43ac-4311-bd4d-abb931cfcc69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:02.983610 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f65daff-43ac-4311-bd4d-abb931cfcc69 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:02.983896 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f65daff-43ac-4311-bd4d-abb931cfcc69 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:02.984175 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f65daff-43ac-4311-bd4d-abb931cfcc69 None admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:02.984990 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f65daff-43ac-4311-bd4d-abb931cfcc69 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:02.985307 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f65daff-43ac-4311-bd4d-abb931cfcc69 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:02.986837 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f65daff-43ac-4311-bd4d-abb931cfcc69 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:03.004046 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 410/828] 10.222.0.58 () {56 vars in 1148 bytes} [Tue Jul 26 13:34:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:03.004046 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f65daff-43ac-4311-bd4d-abb931cfcc69 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:03.066516 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453a7994-275b-49b0-90fa-f86e02aefc91 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:03.106562 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc82dfe-d7bd-4cf5-9800-98467dcb0e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:03.107632 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc82dfe-d7bd-4cf5-9800-98467dcb0e5e None admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:03.108358 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc82dfe-d7bd-4cf5-9800-98467dcb0e5e None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:03.109058 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc82dfe-d7bd-4cf5-9800-98467dcb0e5e None admin] PATH_INFO: `/v3/endpoints` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:03.109968 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc82dfe-d7bd-4cf5-9800-98467dcb0e5e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:03.110671 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f65daff-43ac-4311-bd4d-abb931cfcc69 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:03.111783 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc82dfe-d7bd-4cf5-9800-98467dcb0e5e None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:03.134312 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 419/829] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 13:34:02 2022] POST /identity/v3/endpoints => generated 327 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:34:03.135907 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2937f6-536c-4702-baf3-21041f52c446 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:03.136404 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2937f6-536c-4702-baf3-21041f52c446 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:03.136860 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2937f6-536c-4702-baf3-21041f52c446 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:03.138430 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 420/830] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:34:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:34:03.139394 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2937f6-536c-4702-baf3-21041f52c446 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:03.167919 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6194bf21-c1a8-43e2-a5d3-b05527564a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:03.168677 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6194bf21-c1a8-43e2-a5d3-b05527564a93 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:03.168965 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6194bf21-c1a8-43e2-a5d3-b05527564a93 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:03.169241 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6194bf21-c1a8-43e2-a5d3-b05527564a93 None admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:03.170167 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6194bf21-c1a8-43e2-a5d3-b05527564a93 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:03.170480 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6194bf21-c1a8-43e2-a5d3-b05527564a93 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:03.171951 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6194bf21-c1a8-43e2-a5d3-b05527564a93 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:03.192555 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 411/831] 10.222.0.58 () {56 vars in 1148 bytes} [Tue Jul 26 13:34:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:03.194184 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb554ac7-87ea-45da-a2a6-b03f6d01f42a None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:03.194481 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb554ac7-87ea-45da-a2a6-b03f6d01f42a None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:03.194763 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb554ac7-87ea-45da-a2a6-b03f6d01f42a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:03.215525 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-cb554ac7-87ea-45da-a2a6-b03f6d01f42a None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:34:03.216868 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-cb554ac7-87ea-45da-a2a6-b03f6d01f42a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:34:03.229179 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:03.234498 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:03.281233 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2937f6-536c-4702-baf3-21041f52c446 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:03.312392 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 412/832] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:34:03 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 13:34:03.313788 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb554ac7-87ea-45da-a2a6-b03f6d01f42a None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:03.343469 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635b6e17-287b-4db2-821a-1ca8c637faef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:03.344245 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635b6e17-287b-4db2-821a-1ca8c637faef None admin] REQUEST_METHOD: `PUT` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:03.344533 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635b6e17-287b-4db2-821a-1ca8c637faef None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:03.344832 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635b6e17-287b-4db2-821a-1ca8c637faef None admin] PATH_INFO: `/v3/projects/1e8bb4385c204210a1bc77b1f0d654f3/users/b64bd7b183d24d8c9037b261c7884810/roles/17e0f3c73a8a4a54b3e7b78878dfbf09` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:03.363813 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635b6e17-287b-4db2-821a-1ca8c637faef None admin] RBAC: Authorizing `identity:create_grant(project_id=1e8bb4385c204210a1bc77b1f0d654f3, user_id=b64bd7b183d24d8c9037b261c7884810, role_id=17e0f3c73a8a4a54b3e7b78878dfbf09)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:03.365257 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635b6e17-287b-4db2-821a-1ca8c637faef None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:03.382067 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 421/833] 10.222.0.58 () {58 vars in 1439 bytes} [Tue Jul 26 13:34:03 2022] PUT /identity/v3/projects/1e8bb4385c204210a1bc77b1f0d654f3/users/b64bd7b183d24d8c9037b261c7884810/roles/17e0f3c73a8a4a54b3e7b78878dfbf09 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:34:03.382941 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635b6e17-287b-4db2-821a-1ca8c637faef None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:03.444783 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb554ac7-87ea-45da-a2a6-b03f6d01f42a None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:03.491473 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65def798-3406-4319-88df-0032153b2c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:03.492792 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65def798-3406-4319-88df-0032153b2c58 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:03.493077 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65def798-3406-4319-88df-0032153b2c58 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:03.493458 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65def798-3406-4319-88df-0032153b2c58 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:03.494385 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65def798-3406-4319-88df-0032153b2c58 None admin] RBAC: Adding query filter params (role.id=6e0201a9a6484f7d8b112b1eafff70bd, scope.project.id=1e8bb4385c204210a1bc77b1f0d654f3, user.id=c69161cd520b4b29bf50f3bf2f1c033b) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:03.494710 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65def798-3406-4319-88df-0032153b2c58 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:03.496406 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65def798-3406-4319-88df-0032153b2c58 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:03.499526 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635b6e17-287b-4db2-821a-1ca8c637faef None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:03.504810 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 413/834] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Jul 26 13:34:03 2022] GET /identity/v3/role_assignments?user.id=c69161cd520b4b29bf50f3bf2f1c033b&scope.project.id=1e8bb4385c204210a1bc77b1f0d654f3&role.id=6e0201a9a6484f7d8b112b1eafff70bd => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:03.523118 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:03.541882 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be854792-e810-4ad7-b0ae-37d69bc88f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:03.542575 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be854792-e810-4ad7-b0ae-37d69bc88f89 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:03.542866 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be854792-e810-4ad7-b0ae-37d69bc88f89 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:03.543145 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be854792-e810-4ad7-b0ae-37d69bc88f89 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:03.557405 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be854792-e810-4ad7-b0ae-37d69bc88f89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:34:03.563140 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-be854792-e810-4ad7-b0ae-37d69bc88f89 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 13:34:03.563140 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:03.563140 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:34:03.563140 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:34:03.563140 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:34:03.563140 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:34:03.563140 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:34:03.563140 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:34:03.563140 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:34:03.563140 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:34:03.563140 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:34:03.563140 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:34:03.563140 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:34:03.563140 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:34:03.563140 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:34:03.563140 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:34:03.564128 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:34:03.564128 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:34:03.564128 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:34:03.564128 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:34:03.564128 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:34:03.564128 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:34:03.564128 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:34:03.564128 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:34:03.564128 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:34:03.564128 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:34:03.564128 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:34:03.564128 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:34:03.564128 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:34:03.564128 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:34:03.564128 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:34:03.564128 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:34:03.564969 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:34:03.564969 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:34:03.564969 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:34:03.564969 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:34:03.564969 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:34:03.564969 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:34:03.564969 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:34:03.564969 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:34:03.564969 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:34:03.564969 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:34:03.564969 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 13:34:03.564969 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:34:03.564969 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 422/835] 10.222.0.58 () {56 vars in 1096 bytes} [Tue Jul 26 13:34:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:34:03.571881 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65def798-3406-4319-88df-0032153b2c58 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:03.593454 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:03.703923 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65def798-3406-4319-88df-0032153b2c58 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:03.741373 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc8165f-4670-480d-b86b-c12816c8bc45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:03.742063 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc8165f-4670-480d-b86b-c12816c8bc45 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:03.742353 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc8165f-4670-480d-b86b-c12816c8bc45 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:03.742629 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc8165f-4670-480d-b86b-c12816c8bc45 None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:03.743350 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc8165f-4670-480d-b86b-c12816c8bc45 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:03.743659 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc8165f-4670-480d-b86b-c12816c8bc45 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:03.745082 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc8165f-4670-480d-b86b-c12816c8bc45 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:03.754970 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 414/836] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Jul 26 13:34:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:03.761053 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be854792-e810-4ad7-b0ae-37d69bc88f89 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:03.867076 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be854792-e810-4ad7-b0ae-37d69bc88f89 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:03.906069 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33375292-2ac2-44af-a76d-f96caac153f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:03.906723 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33375292-2ac2-44af-a76d-f96caac153f2 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:03.907006 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33375292-2ac2-44af-a76d-f96caac153f2 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:03.907285 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33375292-2ac2-44af-a76d-f96caac153f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:03.918574 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33375292-2ac2-44af-a76d-f96caac153f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:03.920021 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33375292-2ac2-44af-a76d-f96caac153f2 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:03.935976 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-33375292-2ac2-44af-a76d-f96caac153f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:34:03.935976 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:03.935976 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:34:03.935976 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:34:03.935976 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:34:03.935976 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:34:03.935976 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:34:03.935976 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:34:03.935976 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:34:03.935976 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:34:03.935976 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:34:03.935976 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:34:03.935976 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 13:34:03.935976 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 13:34:03.935976 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 13:34:03.937061 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 13:34:03.937061 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:34:03.937061 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:34:03.937061 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:34:03.937061 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:34:03.937061 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:34:03.937061 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:34:03.937061 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:34:03.937061 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:34:03.937061 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:34:03.937061 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:34:03.937061 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:34:03.937061 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:34:03.937061 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:34:03.937061 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:34:03.937061 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:34:03.937061 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:34:03.938199 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 13:34:03.938199 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 13:34:03.938199 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:34:03.938199 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:34:03.938199 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 423/837] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:34:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:34:03.943190 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc8165f-4670-480d-b86b-c12816c8bc45 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:04.018568 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:04.057049 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc8165f-4670-480d-b86b-c12816c8bc45 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:04.097520 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce5d8e5-c22d-477a-b9ee-ebe7e1c72770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:04.098402 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce5d8e5-c22d-477a-b9ee-ebe7e1c72770 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:04.098866 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce5d8e5-c22d-477a-b9ee-ebe7e1c72770 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:04.099293 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce5d8e5-c22d-477a-b9ee-ebe7e1c72770 None admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:04.100141 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce5d8e5-c22d-477a-b9ee-ebe7e1c72770 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:04.100597 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce5d8e5-c22d-477a-b9ee-ebe7e1c72770 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:04.102175 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce5d8e5-c22d-477a-b9ee-ebe7e1c72770 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:04.115215 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 415/838] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Jul 26 13:34:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:04.120802 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33375292-2ac2-44af-a76d-f96caac153f2 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:04.164089 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:04.233042 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33375292-2ac2-44af-a76d-f96caac153f2 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:04.268537 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364324d5-7f6f-43fd-bd0e-546a904bf20d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:04.269193 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364324d5-7f6f-43fd-bd0e-546a904bf20d None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:04.269493 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364324d5-7f6f-43fd-bd0e-546a904bf20d None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:04.269772 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364324d5-7f6f-43fd-bd0e-546a904bf20d None admin] PATH_INFO: `/v3/users/swift` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:04.278573 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364324d5-7f6f-43fd-bd0e-546a904bf20d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:04.280186 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364324d5-7f6f-43fd-bd0e-546a904bf20d None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:04.292001 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-364324d5-7f6f-43fd-bd0e-546a904bf20d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 26 13:34:04.292001 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:04.292001 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:34:04.292001 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:34:04.292001 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:34:04.292001 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:34:04.292001 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:34:04.292001 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:34:04.292001 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:34:04.292001 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:34:04.292001 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:34:04.292001 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:34:04.292001 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 13:34:04.292001 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 13:34:04.292001 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 13:34:04.292001 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 13:34:04.292960 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:34:04.292960 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:34:04.292960 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 13:34:04.292960 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 13:34:04.292960 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 13:34:04.292960 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 13:34:04.292960 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:34:04.292960 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:34:04.292960 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:34:04.292960 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:34:04.292960 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:34:04.292960 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:34:04.292960 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:34:04.292960 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:34:04.292960 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:34:04.292960 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:34:04.292960 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:34:04.292960 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:34:04.293917 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:34:04.293917 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:34:04.293917 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 13:34:04.293917 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 13:34:04.293917 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 13:34:04.293917 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 13:34:04.293917 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 13:34:04.293917 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 13:34:04.293917 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 26 13:34:04.293917 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:34:04.296898 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 424/839] 10.222.0.58 () {56 vars in 1096 bytes} [Tue Jul 26 13:34:04 2022] GET /identity/v3/users/swift => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:34:04.299623 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce5d8e5-c22d-477a-b9ee-ebe7e1c72770 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:04.406853 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce5d8e5-c22d-477a-b9ee-ebe7e1c72770 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:04.437603 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:04.443860 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cd53d6-1ac3-4f9e-a1a8-5f9b73a48c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:04.444877 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cd53d6-1ac3-4f9e-a1a8-5f9b73a48c71 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:04.445958 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cd53d6-1ac3-4f9e-a1a8-5f9b73a48c71 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:04.446587 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cd53d6-1ac3-4f9e-a1a8-5f9b73a48c71 None admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:04.447611 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cd53d6-1ac3-4f9e-a1a8-5f9b73a48c71 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:04.448353 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cd53d6-1ac3-4f9e-a1a8-5f9b73a48c71 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:04.449788 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cd53d6-1ac3-4f9e-a1a8-5f9b73a48c71 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:04.467441 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 416/840] 10.222.0.58 () {56 vars in 1135 bytes} [Tue Jul 26 13:34:04 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:04.473078 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364324d5-7f6f-43fd-bd0e-546a904bf20d None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:04.569338 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:04.580852 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364324d5-7f6f-43fd-bd0e-546a904bf20d None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:04.619198 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb3c6eb-a266-4bd5-94f9-2bd74511d856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:04.619831 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb3c6eb-a266-4bd5-94f9-2bd74511d856 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:04.620188 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb3c6eb-a266-4bd5-94f9-2bd74511d856 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:04.620474 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb3c6eb-a266-4bd5-94f9-2bd74511d856 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:04.649042 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a861da4-bb8e-40f5-aa34-1152621ac554 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:04.649594 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a861da4-bb8e-40f5-aa34-1152621ac554 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:04.649594 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a861da4-bb8e-40f5-aa34-1152621ac554 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:04.651175 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 417/841] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:34:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:34:04.656244 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff503bb0-3754-4c84-b392-a0636773bf11 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:04.656531 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff503bb0-3754-4c84-b392-a0636773bf11 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:04.656807 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff503bb0-3754-4c84-b392-a0636773bf11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:04.675261 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-ff503bb0-3754-4c84-b392-a0636773bf11 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:34:04.676388 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-ff503bb0-3754-4c84-b392-a0636773bf11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:34:04.696751 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb3c6eb-a266-4bd5-94f9-2bd74511d856 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:04.698365 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb3c6eb-a266-4bd5-94f9-2bd74511d856 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:04.757810 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 418/842] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:34:04 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 13:34:04.773924 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff503bb0-3754-4c84-b392-a0636773bf11 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:04.800809 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:04.817576 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 425/843] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Jul 26 13:34:04 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 13:34:04.821629 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb3c6eb-a266-4bd5-94f9-2bd74511d856 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:04.898426 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff503bb0-3754-4c84-b392-a0636773bf11 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:04.917957 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:04.924121 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb3c6eb-a266-4bd5-94f9-2bd74511d856 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:04.942531 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcef1c01-3f8c-4df8-9b36-3c208da148cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:04.943036 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcef1c01-3f8c-4df8-9b36-3c208da148cd None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:04.943326 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcef1c01-3f8c-4df8-9b36-3c208da148cd None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:04.943603 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcef1c01-3f8c-4df8-9b36-3c208da148cd None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:04.944238 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcef1c01-3f8c-4df8-9b36-3c208da148cd None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:04.946874 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcef1c01-3f8c-4df8-9b36-3c208da148cd None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:04.958589 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-dcef1c01-3f8c-4df8-9b36-3c208da148cd None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 26 13:34:04.958589 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:04.958589 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 13:34:04.958589 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 13:34:04.958589 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 13:34:04.958589 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 13:34:04.958589 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 13:34:04.958589 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 13:34:04.958589 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 26 13:34:04.958589 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:34:04.958589 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 13:34:04.958589 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:34:04.958589 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:04.958589 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:34:04.958589 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:34:04.958589 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:34:04.958589 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:34:04.959784 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:34:04.959784 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:34:04.959784 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:34:04.959784 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:34:04.959784 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:34:04.959784 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:34:04.959784 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 13:34:04.959784 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 13:34:04.959784 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 13:34:04.959784 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 13:34:04.959784 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:34:04.959784 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:34:04.959784 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:34:04.959784 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:34:04.959784 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:34:04.959784 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:34:04.961330 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:34:04.961330 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:34:04.961330 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:34:04.961330 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:34:04.961330 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:34:04.961330 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:34:04.961330 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:34:04.961330 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:34:04.961330 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:34:04.961330 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:34:04.961330 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 13:34:04.961330 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 13:34:04.961330 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 26 13:34:04.961330 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:34:04.961330 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 419/844] 10.222.0.58 () {56 vars in 1114 bytes} [Tue Jul 26 13:34:04 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:34:04.963997 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc790d02-dce8-4c29-b3a3-d50bc164b04a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:04.964650 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc790d02-dce8-4c29-b3a3-d50bc164b04a None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:04.964936 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc790d02-dce8-4c29-b3a3-d50bc164b04a None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:04.965214 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc790d02-dce8-4c29-b3a3-d50bc164b04a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:04.966428 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcef1c01-3f8c-4df8-9b36-3c208da148cd None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:04.979719 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc790d02-dce8-4c29-b3a3-d50bc164b04a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:04.981685 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc790d02-dce8-4c29-b3a3-d50bc164b04a None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:05.000470 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-fc790d02-dce8-4c29-b3a3-d50bc164b04a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:34:05.000470 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:05.000470 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:34:05.000470 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:34:05.000470 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:34:05.000470 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:34:05.000470 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:34:05.000470 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:34:05.000470 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:34:05.000470 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:34:05.000470 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:34:05.000470 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:34:05.000470 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 13:34:05.000470 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 13:34:05.000470 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 13:34:05.001386 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 13:34:05.001386 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:34:05.001386 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:34:05.001386 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:34:05.001386 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:34:05.001386 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:34:05.001386 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:34:05.001386 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:34:05.001386 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:34:05.001386 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:34:05.001386 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:34:05.001386 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:34:05.001386 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:34:05.001386 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:34:05.001386 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:34:05.001386 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:34:05.001386 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:34:05.002628 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 13:34:05.002628 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 13:34:05.002628 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:34:05.002628 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:34:05.005648 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 426/845] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:34:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:34:05.009040 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc790d02-dce8-4c29-b3a3-d50bc164b04a None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:05.062541 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:05.096242 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcef1c01-3f8c-4df8-9b36-3c208da148cd None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:05.121851 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc790d02-dce8-4c29-b3a3-d50bc164b04a None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:05.132325 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a5eaa6-2205-4a95-a4e6-cdc96722507d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:05.133537 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a5eaa6-2205-4a95-a4e6-cdc96722507d None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:05.133776 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a5eaa6-2205-4a95-a4e6-cdc96722507d None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:05.134055 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a5eaa6-2205-4a95-a4e6-cdc96722507d None admin] PATH_INFO: `/v3/services` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:05.134743 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a5eaa6-2205-4a95-a4e6-cdc96722507d None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:05.135052 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a5eaa6-2205-4a95-a4e6-cdc96722507d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:05.136422 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a5eaa6-2205-4a95-a4e6-cdc96722507d None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:05.141826 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 420/846] 10.222.0.58 () {56 vars in 1114 bytes} [Tue Jul 26 13:34:04 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:34:05.151180 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a5eaa6-2205-4a95-a4e6-cdc96722507d None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:05.162114 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca33490b-e85e-4195-a38d-8537137fd5df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:05.162884 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca33490b-e85e-4195-a38d-8537137fd5df None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:05.163273 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca33490b-e85e-4195-a38d-8537137fd5df None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:05.163652 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca33490b-e85e-4195-a38d-8537137fd5df None admin] PATH_INFO: `/v3/domains` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:05.164491 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca33490b-e85e-4195-a38d-8537137fd5df None admin] RBAC: Adding query filter params (name=Default) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:05.164933 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca33490b-e85e-4195-a38d-8537137fd5df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:05.166440 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca33490b-e85e-4195-a38d-8537137fd5df None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:05.179441 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 427/847] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Jul 26 13:34:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:05.189081 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca33490b-e85e-4195-a38d-8537137fd5df None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:05.208797 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:05.270662 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a5eaa6-2205-4a95-a4e6-cdc96722507d None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:05.311592 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d4b744-2941-4ab7-9f87-7c3dfdb0a884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:05.312208 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d4b744-2941-4ab7-9f87-7c3dfdb0a884 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:05.312493 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d4b744-2941-4ab7-9f87-7c3dfdb0a884 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:05.312771 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d4b744-2941-4ab7-9f87-7c3dfdb0a884 None admin] PATH_INFO: `/v3/services` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:05.313456 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d4b744-2941-4ab7-9f87-7c3dfdb0a884 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:05.313791 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d4b744-2941-4ab7-9f87-7c3dfdb0a884 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:05.315152 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d4b744-2941-4ab7-9f87-7c3dfdb0a884 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:05.319809 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 421/848] 10.222.0.58 () {56 vars in 1114 bytes} [Tue Jul 26 13:34:05 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:05.325789 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca33490b-e85e-4195-a38d-8537137fd5df None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:05.328581 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:05.367668 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbcc8f1-e782-4cc6-9d47-1eee197683b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:05.368526 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbcc8f1-e782-4cc6-9d47-1eee197683b6 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:05.368882 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbcc8f1-e782-4cc6-9d47-1eee197683b6 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:05.369233 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbcc8f1-e782-4cc6-9d47-1eee197683b6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:05.377676 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbcc8f1-e782-4cc6-9d47-1eee197683b6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:05.379242 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbcc8f1-e782-4cc6-9d47-1eee197683b6 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:05.392923 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-9dbcc8f1-e782-4cc6-9d47-1eee197683b6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 13:34:05.392923 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:05.392923 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:34:05.392923 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:34:05.392923 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:34:05.392923 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:34:05.392923 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:34:05.392923 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:34:05.392923 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:34:05.392923 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:34:05.392923 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:34:05.392923 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:34:05.392923 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 13:34:05.392923 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 13:34:05.392923 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 13:34:05.393945 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 13:34:05.393945 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:34:05.393945 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:34:05.393945 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:34:05.393945 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:34:05.393945 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:34:05.393945 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:34:05.393945 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:34:05.393945 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:34:05.393945 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:34:05.393945 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:34:05.393945 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:34:05.393945 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:34:05.393945 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:34:05.393945 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:34:05.393945 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:34:05.393945 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:34:05.394827 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 13:34:05.394827 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 13:34:05.394827 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 13:34:05.394827 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 13:34:05.394827 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 13:34:05.394827 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 13:34:05.394827 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 13:34:05.394827 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:34:05.394827 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 428/849] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Jul 26 13:34:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:34:05.400128 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d4b744-2941-4ab7-9f87-7c3dfdb0a884 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:05.516583 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d4b744-2941-4ab7-9f87-7c3dfdb0a884 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:05.527248 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:05.553591 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a13be27-fcfa-491d-a68d-150b82658280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:05.554625 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a13be27-fcfa-491d-a68d-150b82658280 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:05.555580 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a13be27-fcfa-491d-a68d-150b82658280 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:05.556051 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a13be27-fcfa-491d-a68d-150b82658280 None admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:05.556973 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a13be27-fcfa-491d-a68d-150b82658280 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:05.557453 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a13be27-fcfa-491d-a68d-150b82658280 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:05.559070 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a13be27-fcfa-491d-a68d-150b82658280 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:05.572625 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 422/850] 10.222.0.58 () {56 vars in 1148 bytes} [Tue Jul 26 13:34:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:05.578635 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbcc8f1-e782-4cc6-9d47-1eee197683b6 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:05.598994 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48027591-5ae3-478a-821e-ea526dbe925d None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:05.599324 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48027591-5ae3-478a-821e-ea526dbe925d None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:05.599592 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48027591-5ae3-478a-821e-ea526dbe925d None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:05.601055 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 423/851] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:34:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:34:05.606840 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964632c3-3ae2-4d08-adb0-c250b01d7b82 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:05.607355 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964632c3-3ae2-4d08-adb0-c250b01d7b82 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:05.607828 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964632c3-3ae2-4d08-adb0-c250b01d7b82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:05.627063 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-964632c3-3ae2-4d08-adb0-c250b01d7b82 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:34:05.628668 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-964632c3-3ae2-4d08-adb0-c250b01d7b82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:34:05.692079 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbcc8f1-e782-4cc6-9d47-1eee197683b6 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:05.711332 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 424/852] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:34:05 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 13:34:05.729090 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0855762-c59a-4f2c-b44f-48d326f51dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:05.729768 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0855762-c59a-4f2c-b44f-48d326f51dfd None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:05.730054 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0855762-c59a-4f2c-b44f-48d326f51dfd None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:05.730336 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0855762-c59a-4f2c-b44f-48d326f51dfd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:05.731155 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0855762-c59a-4f2c-b44f-48d326f51dfd None admin] RBAC: Adding query filter params (role.id=6e0201a9a6484f7d8b112b1eafff70bd, scope.project.id=1e8bb4385c204210a1bc77b1f0d654f3, user.id=3ab2819bac844163ae4305e4f2186605) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:05.731471 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0855762-c59a-4f2c-b44f-48d326f51dfd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:05.733101 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0855762-c59a-4f2c-b44f-48d326f51dfd None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:05.737681 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964632c3-3ae2-4d08-adb0-c250b01d7b82 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:05.739820 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 429/853] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Jul 26 13:34:05 2022] GET /identity/v3/role_assignments?user.id=3ab2819bac844163ae4305e4f2186605&scope.project.id=1e8bb4385c204210a1bc77b1f0d654f3&role.id=6e0201a9a6484f7d8b112b1eafff70bd => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:05.795131 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2348250-e988-4b6c-b57e-a418d6fa3548 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:05.795439 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2348250-e988-4b6c-b57e-a418d6fa3548 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:05.795718 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2348250-e988-4b6c-b57e-a418d6fa3548 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:05.797538 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 430/854] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:34:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 13:34:05.803274 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49d7d0a-3964-4530-a644-36db3935832b None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:05.803895 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49d7d0a-3964-4530-a644-36db3935832b None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:05.804105 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49d7d0a-3964-4530-a644-36db3935832b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:05.824884 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-b49d7d0a-3964-4530-a644-36db3935832b None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:34:05.826147 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-b49d7d0a-3964-4530-a644-36db3935832b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:34:05.861058 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:05.871672 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:05.876928 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964632c3-3ae2-4d08-adb0-c250b01d7b82 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:05.925890 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 431/855] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:34:05 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 13:34:05.929049 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f9424b-ab13-4273-a79b-7bd1c656e6cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:05.930343 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f9424b-ab13-4273-a79b-7bd1c656e6cc None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:05.930619 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f9424b-ab13-4273-a79b-7bd1c656e6cc None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:05.930899 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f9424b-ab13-4273-a79b-7bd1c656e6cc None admin] PATH_INFO: `/v3/roles/service` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:05.942305 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f9424b-ab13-4273-a79b-7bd1c656e6cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:34:05.944344 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49d7d0a-3964-4530-a644-36db3935832b None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:05.948628 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-09f9424b-ab13-4273-a79b-7bd1c656e6cc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 13:34:05.948628 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:05.948628 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:34:05.948628 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:34:05.948628 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:34:05.948628 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:34:05.948628 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:34:05.948628 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:34:05.948628 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:34:05.948628 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:34:05.948628 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:34:05.948628 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:34:05.948628 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:34:05.948628 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:34:05.948628 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:34:05.948628 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:34:05.949596 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:34:05.949596 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:34:05.949596 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:34:05.949596 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:34:05.949596 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:34:05.949596 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:34:05.949596 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:34:05.949596 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:34:05.949596 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:34:05.949596 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:34:05.949596 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:34:05.949596 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:34:05.949596 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:34:05.949596 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:34:05.949596 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:34:05.949596 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:34:05.950457 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:34:05.950457 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:34:05.950457 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:34:05.950457 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:34:05.950457 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:34:05.950457 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:34:05.950457 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:34:05.950457 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:34:05.950457 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:34:05.950457 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:34:05.950457 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 13:34:05.950457 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:34:05.950457 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 425/856] 10.222.0.58 () {56 vars in 1102 bytes} [Tue Jul 26 13:34:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:34:05.957131 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f9424b-ab13-4273-a79b-7bd1c656e6cc None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:06.084811 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49d7d0a-3964-4530-a644-36db3935832b None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:06.109527 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f9424b-ab13-4273-a79b-7bd1c656e6cc None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:06.134413 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da8a8c0-9395-45ca-b932-b1a885204cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:06.135365 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da8a8c0-9395-45ca-b932-b1a885204cfd None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:06.135795 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da8a8c0-9395-45ca-b932-b1a885204cfd None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:06.136214 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da8a8c0-9395-45ca-b932-b1a885204cfd None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:06.137015 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da8a8c0-9395-45ca-b932-b1a885204cfd None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:06.138652 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da8a8c0-9395-45ca-b932-b1a885204cfd None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:06.150944 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-1da8a8c0-9395-45ca-b932-b1a885204cfd None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 26 13:34:06.150944 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:06.150944 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 13:34:06.150944 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 13:34:06.150944 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 13:34:06.150944 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 13:34:06.150944 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 13:34:06.150944 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 13:34:06.150944 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 26 13:34:06.150944 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:34:06.150944 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 13:34:06.150944 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:34:06.150944 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:06.150944 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:34:06.150944 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:34:06.150944 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:34:06.152166 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:34:06.152166 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:34:06.152166 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:34:06.152166 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:34:06.152166 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:34:06.152166 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:34:06.152166 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:34:06.152166 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 13:34:06.152166 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 13:34:06.152166 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 13:34:06.152166 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 13:34:06.152166 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:34:06.152166 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:34:06.152166 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:34:06.152166 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:34:06.152166 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:34:06.153158 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:34:06.153158 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:34:06.153158 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:34:06.153158 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:34:06.153158 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:34:06.153158 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:34:06.153158 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:34:06.153158 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:34:06.153158 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:34:06.153158 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:34:06.153158 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:34:06.153158 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 13:34:06.153158 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 13:34:06.153158 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 26 13:34:06.153158 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:34:06.155958 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2504f1-b934-4987-a8c4-680d392f6b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:06.156676 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2504f1-b934-4987-a8c4-680d392f6b1f None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:06.156885 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2504f1-b934-4987-a8c4-680d392f6b1f None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:06.157159 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2504f1-b934-4987-a8c4-680d392f6b1f None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:06.157302 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 432/857] 10.222.0.58 () {56 vars in 1132 bytes} [Tue Jul 26 13:34:05 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 13:34:06.163040 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2504f1-b934-4987-a8c4-680d392f6b1f None admin] RBAC: Adding query filter params (name=service) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:06.163040 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2504f1-b934-4987-a8c4-680d392f6b1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:06.163040 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2504f1-b934-4987-a8c4-680d392f6b1f None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:06.165858 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da8a8c0-9395-45ca-b932-b1a885204cfd None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:06.177617 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 426/858] 10.222.0.58 () {56 vars in 1103 bytes} [Tue Jul 26 13:34:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:34:06.179908 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2504f1-b934-4987-a8c4-680d392f6b1f None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:06.200377 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:06.237614 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:06.305069 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da8a8c0-9395-45ca-b932-b1a885204cfd None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:06.363978 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881bcf76-c51e-4707-b872-7c9aa8ba5e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:06.364830 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881bcf76-c51e-4707-b872-7c9aa8ba5e61 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:06.365260 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881bcf76-c51e-4707-b872-7c9aa8ba5e61 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:06.365759 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881bcf76-c51e-4707-b872-7c9aa8ba5e61 None admin] PATH_INFO: `/v3/services` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:06.366410 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2504f1-b934-4987-a8c4-680d392f6b1f None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:06.366818 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881bcf76-c51e-4707-b872-7c9aa8ba5e61 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:06.367344 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881bcf76-c51e-4707-b872-7c9aa8ba5e61 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:06.371590 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881bcf76-c51e-4707-b872-7c9aa8ba5e61 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:06.381946 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 433/859] 10.222.0.58 () {56 vars in 1126 bytes} [Tue Jul 26 13:34:06 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:06.388003 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881bcf76-c51e-4707-b872-7c9aa8ba5e61 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:06.412313 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0178df-961f-497f-a1b7-c56dfabc9e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:06.413438 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0178df-961f-497f-a1b7-c56dfabc9e23 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:06.413999 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0178df-961f-497f-a1b7-c56dfabc9e23 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:06.414531 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0178df-961f-497f-a1b7-c56dfabc9e23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:06.427021 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0178df-961f-497f-a1b7-c56dfabc9e23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:06.428810 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0178df-961f-497f-a1b7-c56dfabc9e23 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:06.444405 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-6b0178df-961f-497f-a1b7-c56dfabc9e23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:34:06.444405 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:06.444405 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:34:06.444405 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:34:06.444405 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:34:06.444405 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:34:06.444405 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:34:06.444405 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:34:06.444405 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:34:06.444405 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:34:06.444405 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:34:06.444405 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:34:06.444405 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 13:34:06.444405 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 13:34:06.444405 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 13:34:06.445940 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 13:34:06.445940 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:34:06.445940 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:34:06.445940 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:34:06.445940 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:34:06.445940 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:34:06.445940 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:34:06.445940 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:34:06.445940 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:34:06.445940 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:34:06.445940 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:34:06.445940 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:34:06.445940 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:34:06.445940 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:34:06.445940 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:34:06.445940 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:34:06.445940 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:34:06.446829 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 13:34:06.446829 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 13:34:06.446829 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:34:06.446829 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:34:06.449177 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 427/860] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:34:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:34:06.454391 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0178df-961f-497f-a1b7-c56dfabc9e23 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:06.494284 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:06.512827 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881bcf76-c51e-4707-b872-7c9aa8ba5e61 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:06.537255 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:06.552518 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dbc3aa-56a4-4dae-9d8f-d2eed8775765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:06.553016 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dbc3aa-56a4-4dae-9d8f-d2eed8775765 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:06.553307 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dbc3aa-56a4-4dae-9d8f-d2eed8775765 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:06.553619 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dbc3aa-56a4-4dae-9d8f-d2eed8775765 None admin] PATH_INFO: `/v3/services` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:06.554317 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dbc3aa-56a4-4dae-9d8f-d2eed8775765 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:06.554663 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dbc3aa-56a4-4dae-9d8f-d2eed8775765 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:06.556056 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dbc3aa-56a4-4dae-9d8f-d2eed8775765 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:06.569510 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 434/861] 10.222.0.58 () {56 vars in 1126 bytes} [Tue Jul 26 13:34:06 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:34:06.576492 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0178df-961f-497f-a1b7-c56dfabc9e23 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:06.617430 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0073a751-2324-4d7f-a678-adce701506de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:06.618021 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0073a751-2324-4d7f-a678-adce701506de None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:06.618308 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0073a751-2324-4d7f-a678-adce701506de None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:06.618630 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0073a751-2324-4d7f-a678-adce701506de None admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:06.619347 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0073a751-2324-4d7f-a678-adce701506de None admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:06.619653 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0073a751-2324-4d7f-a678-adce701506de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:06.621081 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0073a751-2324-4d7f-a678-adce701506de None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:06.637663 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 428/862] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Jul 26 13:34:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:06.643795 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dbc3aa-56a4-4dae-9d8f-d2eed8775765 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:06.765024 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dbc3aa-56a4-4dae-9d8f-d2eed8775765 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:06.800985 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1feb03-e3f9-4daf-ac5b-2015f19019b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:06.801747 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1feb03-e3f9-4daf-ac5b-2015f19019b6 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:06.802033 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1feb03-e3f9-4daf-ac5b-2015f19019b6 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:06.802351 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1feb03-e3f9-4daf-ac5b-2015f19019b6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:06.811873 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1feb03-e3f9-4daf-ac5b-2015f19019b6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:06.813323 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1feb03-e3f9-4daf-ac5b-2015f19019b6 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:06.826096 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-5b1feb03-e3f9-4daf-ac5b-2015f19019b6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 13:34:06.826096 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:06.826096 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:34:06.826096 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:34:06.826096 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:34:06.826096 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:34:06.826096 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:34:06.826096 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:34:06.826096 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:34:06.826096 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:34:06.826096 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:34:06.826096 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:34:06.826096 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 13:34:06.826096 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 13:34:06.826096 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 13:34:06.826096 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 13:34:06.827091 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:34:06.827091 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:34:06.827091 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 13:34:06.827091 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 13:34:06.827091 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 13:34:06.827091 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 13:34:06.827091 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:34:06.827091 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:34:06.827091 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:34:06.827091 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:34:06.827091 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:34:06.827091 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:34:06.827091 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:34:06.827091 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:34:06.827091 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:34:06.827091 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:34:06.827091 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:34:06.827091 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:34:06.828003 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:34:06.828003 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:34:06.828003 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 13:34:06.828003 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 13:34:06.828003 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 13:34:06.828003 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 13:34:06.828003 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 13:34:06.828003 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 13:34:06.828003 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 13:34:06.828003 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:34:06.828003 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 435/863] 10.222.0.58 () {56 vars in 1117 bytes} [Tue Jul 26 13:34:06 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:34:06.833633 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0073a751-2324-4d7f-a678-adce701506de None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:06.941827 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0073a751-2324-4d7f-a678-adce701506de None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:06.978472 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b7bd17-571c-4d93-9050-1de64e5aeb25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:06.979494 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b7bd17-571c-4d93-9050-1de64e5aeb25 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:06.979988 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b7bd17-571c-4d93-9050-1de64e5aeb25 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:06.980500 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b7bd17-571c-4d93-9050-1de64e5aeb25 None admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:06.981572 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b7bd17-571c-4d93-9050-1de64e5aeb25 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:06.982105 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b7bd17-571c-4d93-9050-1de64e5aeb25 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:06.983812 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b7bd17-571c-4d93-9050-1de64e5aeb25 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:06.986327 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:07.005321 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 429/864] 10.222.0.58 () {56 vars in 1149 bytes} [Tue Jul 26 13:34:06 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:07.011161 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1feb03-e3f9-4daf-ac5b-2015f19019b6 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:07.026816 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:07.130674 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1feb03-e3f9-4daf-ac5b-2015f19019b6 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:07.166023 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92456377-c403-4392-89f2-5880199416d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:07.166933 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92456377-c403-4392-89f2-5880199416d2 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:07.167364 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92456377-c403-4392-89f2-5880199416d2 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:07.167783 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92456377-c403-4392-89f2-5880199416d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:07.241269 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92456377-c403-4392-89f2-5880199416d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:07.243533 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92456377-c403-4392-89f2-5880199416d2 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:07.266647 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:07.357840 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 436/865] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Jul 26 13:34:07 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 13:34:07.365810 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b7bd17-571c-4d93-9050-1de64e5aeb25 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:07.482546 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b7bd17-571c-4d93-9050-1de64e5aeb25 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:07.520292 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f2095f-dee5-4cb3-9a02-737755531154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:07.521203 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f2095f-dee5-4cb3-9a02-737755531154 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:07.521677 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f2095f-dee5-4cb3-9a02-737755531154 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:07.522162 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f2095f-dee5-4cb3-9a02-737755531154 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:07.534379 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f2095f-dee5-4cb3-9a02-737755531154 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:07.536088 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f2095f-dee5-4cb3-9a02-737755531154 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:07.553427 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-63f2095f-dee5-4cb3-9a02-737755531154 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:34:07.553427 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:07.553427 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:34:07.553427 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:34:07.553427 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:34:07.553427 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:34:07.553427 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:34:07.553427 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:34:07.553427 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:34:07.553427 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:34:07.553427 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:34:07.553427 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:34:07.553427 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 13:34:07.553427 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 13:34:07.553427 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 13:34:07.554638 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 13:34:07.554638 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:34:07.554638 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:34:07.554638 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:34:07.554638 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:34:07.554638 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:34:07.554638 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:34:07.554638 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:34:07.554638 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:34:07.554638 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:34:07.554638 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:34:07.554638 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:34:07.554638 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:34:07.554638 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:34:07.554638 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:34:07.554638 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:34:07.554638 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:34:07.555753 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 13:34:07.555753 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 13:34:07.555753 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:34:07.555753 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:34:07.557612 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 430/866] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:34:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:34:07.566806 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92456377-c403-4392-89f2-5880199416d2 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:07.681005 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92456377-c403-4392-89f2-5880199416d2 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:07.693166 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:07.720666 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808599f7-caf7-42f2-8977-872fc75d3ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:07.721583 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808599f7-caf7-42f2-8977-872fc75d3ec9 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:07.722066 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808599f7-caf7-42f2-8977-872fc75d3ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:07.722489 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808599f7-caf7-42f2-8977-872fc75d3ec9 None admin] PATH_INFO: `/v3/domains` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:07.723399 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808599f7-caf7-42f2-8977-872fc75d3ec9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:07.723857 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808599f7-caf7-42f2-8977-872fc75d3ec9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:07.725536 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808599f7-caf7-42f2-8977-872fc75d3ec9 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:07.739488 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 437/867] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Jul 26 13:34:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:07.746610 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f2095f-dee5-4cb3-9a02-737755531154 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:07.812281 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:07.869180 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8b85f6-3293-4b95-97df-1d7682eaae75 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:07.870837 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8b85f6-3293-4b95-97df-1d7682eaae75 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:07.870837 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8b85f6-3293-4b95-97df-1d7682eaae75 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:07.871792 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 438/868] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:34:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 13:34:07.878370 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894ff323-ced6-4fa7-85bd-9060e281ddf9 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:07.879049 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894ff323-ced6-4fa7-85bd-9060e281ddf9 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:07.879529 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894ff323-ced6-4fa7-85bd-9060e281ddf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:07.893588 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f2095f-dee5-4cb3-9a02-737755531154 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:07.901311 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-894ff323-ced6-4fa7-85bd-9060e281ddf9 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:34:07.902840 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-894ff323-ced6-4fa7-85bd-9060e281ddf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:34:07.934851 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff2141d-c6b0-4eb4-86aa-832539a87cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:07.935885 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff2141d-c6b0-4eb4-86aa-832539a87cb1 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:07.936506 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff2141d-c6b0-4eb4-86aa-832539a87cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:07.937020 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff2141d-c6b0-4eb4-86aa-832539a87cb1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:07.945683 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff2141d-c6b0-4eb4-86aa-832539a87cb1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:07.947129 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff2141d-c6b0-4eb4-86aa-832539a87cb1 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:07.959083 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-5ff2141d-c6b0-4eb4-86aa-832539a87cb1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 13:34:07.959083 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:07.959083 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:34:07.959083 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:34:07.959083 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:34:07.959083 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:34:07.959083 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:34:07.959083 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:34:07.959083 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:34:07.959083 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:34:07.959083 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:34:07.959083 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:34:07.959083 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 13:34:07.959083 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 13:34:07.959083 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 13:34:07.960045 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 13:34:07.960045 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:34:07.960045 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:34:07.960045 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:34:07.960045 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:34:07.960045 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:34:07.960045 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:34:07.960045 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:34:07.960045 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:34:07.960045 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:34:07.960045 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:34:07.960045 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:34:07.960045 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:34:07.960045 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:34:07.960045 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:34:07.960045 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:34:07.960045 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:34:07.962201 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 13:34:07.962201 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 13:34:07.962201 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 13:34:07.962201 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 13:34:07.962201 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 13:34:07.962201 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 13:34:07.962201 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 13:34:07.962201 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:34:07.962201 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 431/869] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Jul 26 13:34:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:34:07.962880 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f368f3-7b38-4c01-8465-3e038511d2cd None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:07.962880 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f368f3-7b38-4c01-8465-3e038511d2cd None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:07.962880 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f368f3-7b38-4c01-8465-3e038511d2cd None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:07.964239 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 432/870] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:34:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:34:07.968147 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f368f3-7b38-4c01-8465-3e038511d2cd None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:07.985922 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 439/871] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:34:07 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 13:34:07.987623 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c2f85d-d7fc-4835-ab65-a6c5b70731c1 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:07.987895 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c2f85d-d7fc-4835-ab65-a6c5b70731c1 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:07.988143 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c2f85d-d7fc-4835-ab65-a6c5b70731c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:08.008031 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-a4c2f85d-d7fc-4835-ab65-a6c5b70731c1 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:34:08.010031 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-a4c2f85d-d7fc-4835-ab65-a6c5b70731c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:34:08.098260 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f368f3-7b38-4c01-8465-3e038511d2cd None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:08.100588 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:08.112435 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 440/872] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:34:07 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 13:34:08.118766 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c2f85d-d7fc-4835-ab65-a6c5b70731c1 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:08.139337 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10abf0cb-dd3d-4d58-8afd-0db80dfc5363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:08.139951 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10abf0cb-dd3d-4d58-8afd-0db80dfc5363 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:08.140229 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10abf0cb-dd3d-4d58-8afd-0db80dfc5363 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:08.140772 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10abf0cb-dd3d-4d58-8afd-0db80dfc5363 None admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:08.141246 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10abf0cb-dd3d-4d58-8afd-0db80dfc5363 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:08.141659 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10abf0cb-dd3d-4d58-8afd-0db80dfc5363 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:08.143073 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10abf0cb-dd3d-4d58-8afd-0db80dfc5363 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:08.164862 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:08.169519 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 433/873] 10.222.0.58 () {56 vars in 1148 bytes} [Tue Jul 26 13:34:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:34:08.169519 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10abf0cb-dd3d-4d58-8afd-0db80dfc5363 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:08.260921 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c2f85d-d7fc-4835-ab65-a6c5b70731c1 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:08.293143 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10abf0cb-dd3d-4d58-8afd-0db80dfc5363 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:08.298612 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976e7cd8-18dd-4944-b603-7c64a070b5d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:08.299247 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976e7cd8-18dd-4944-b603-7c64a070b5d8 None admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:08.299516 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976e7cd8-18dd-4944-b603-7c64a070b5d8 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:08.299785 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976e7cd8-18dd-4944-b603-7c64a070b5d8 None admin] PATH_INFO: `/v3/services` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:08.300477 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976e7cd8-18dd-4944-b603-7c64a070b5d8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:08.301868 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976e7cd8-18dd-4944-b603-7c64a070b5d8 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:08.321511 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 441/874] 10.222.0.58 () {60 vars in 1141 bytes} [Tue Jul 26 13:34:08 2022] POST /identity/v3/services => generated 251 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:34:08.321511 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976e7cd8-18dd-4944-b603-7c64a070b5d8 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:08.349633 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478a2785-4c93-49ff-ac9e-3fe8884cf0dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:08.350320 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478a2785-4c93-49ff-ac9e-3fe8884cf0dd None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:08.350710 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478a2785-4c93-49ff-ac9e-3fe8884cf0dd None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:08.351051 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478a2785-4c93-49ff-ac9e-3fe8884cf0dd None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:08.352016 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478a2785-4c93-49ff-ac9e-3fe8884cf0dd None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:08.353635 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478a2785-4c93-49ff-ac9e-3fe8884cf0dd None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:08.365433 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-478a2785-4c93-49ff-ac9e-3fe8884cf0dd None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 13:34:08.365433 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:08.365433 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 13:34:08.365433 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 13:34:08.365433 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 13:34:08.365433 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 13:34:08.365433 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 13:34:08.365433 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 13:34:08.365433 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 13:34:08.365433 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:34:08.365433 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 13:34:08.365433 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:34:08.365433 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:08.365433 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:34:08.365433 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:34:08.365433 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:34:08.365433 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:34:08.366752 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:34:08.366752 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:34:08.366752 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:34:08.366752 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:34:08.366752 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:34:08.366752 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:34:08.366752 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 13:34:08.366752 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 13:34:08.366752 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 13:34:08.366752 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 13:34:08.366752 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:34:08.366752 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:34:08.366752 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:34:08.366752 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:34:08.366752 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:34:08.366752 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:34:08.368348 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:34:08.368348 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:34:08.368348 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:34:08.368348 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:34:08.368348 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:34:08.368348 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:34:08.368348 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:34:08.368348 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:34:08.368348 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:34:08.368348 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:34:08.368348 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 13:34:08.368348 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 13:34:08.368348 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 13:34:08.368348 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:34:08.375033 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 434/875] 10.222.0.58 () {56 vars in 1126 bytes} [Tue Jul 26 13:34:08 2022] GET /identity/v3/services/object-store => generated 93 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:34:08.376412 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478a2785-4c93-49ff-ac9e-3fe8884cf0dd None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:08.411728 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:08.474216 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:08.488582 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976e7cd8-18dd-4944-b603-7c64a070b5d8 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:08.542759 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c55230-d6fb-425f-bccb-5d71b7096929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:08.543806 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c55230-d6fb-425f-bccb-5d71b7096929 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:08.545897 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c55230-d6fb-425f-bccb-5d71b7096929 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:08.546158 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478a2785-4c93-49ff-ac9e-3fe8884cf0dd None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:08.546371 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c55230-d6fb-425f-bccb-5d71b7096929 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:08.547560 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c55230-d6fb-425f-bccb-5d71b7096929 None admin] RBAC: Adding query filter params (role.id=17e0f3c73a8a4a54b3e7b78878dfbf09, scope.project.id=1e8bb4385c204210a1bc77b1f0d654f3, user.id=b64bd7b183d24d8c9037b261c7884810) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:08.547994 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c55230-d6fb-425f-bccb-5d71b7096929 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:08.550714 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c55230-d6fb-425f-bccb-5d71b7096929 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:08.558855 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 442/876] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Jul 26 13:34:08 2022] GET /identity/v3/role_assignments?user.id=b64bd7b183d24d8c9037b261c7884810&scope.project.id=1e8bb4385c204210a1bc77b1f0d654f3&role.id=17e0f3c73a8a4a54b3e7b78878dfbf09 => generated 608 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:08.592131 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0169d134-8aa5-4312-99f9-8d7c0c929e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:08.593025 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0169d134-8aa5-4312-99f9-8d7c0c929e55 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:08.593318 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0169d134-8aa5-4312-99f9-8d7c0c929e55 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:08.593721 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0169d134-8aa5-4312-99f9-8d7c0c929e55 None admin] PATH_INFO: `/v3/services` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:08.594587 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0169d134-8aa5-4312-99f9-8d7c0c929e55 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:08.594908 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0169d134-8aa5-4312-99f9-8d7c0c929e55 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:08.596499 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0169d134-8aa5-4312-99f9-8d7c0c929e55 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:08.604939 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 435/877] 10.222.0.58 () {56 vars in 1122 bytes} [Tue Jul 26 13:34:08 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:08.611856 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c55230-d6fb-425f-bccb-5d71b7096929 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:08.750936 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c55230-d6fb-425f-bccb-5d71b7096929 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:08.787987 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d760d4f2-299d-491e-bbd9-004f2beb57cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:08.789543 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d760d4f2-299d-491e-bbd9-004f2beb57cd None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:08.790059 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d760d4f2-299d-491e-bbd9-004f2beb57cd None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:08.790540 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d760d4f2-299d-491e-bbd9-004f2beb57cd None admin] PATH_INFO: `/v3/services` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:08.791454 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d760d4f2-299d-491e-bbd9-004f2beb57cd None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:08.791983 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d760d4f2-299d-491e-bbd9-004f2beb57cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:08.793776 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d760d4f2-299d-491e-bbd9-004f2beb57cd None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:08.800208 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 443/878] 10.222.0.58 () {56 vars in 1122 bytes} [Tue Jul 26 13:34:08 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:08.816950 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927c9c5e-7539-47d8-961e-928946fdcac6 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:08.817268 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927c9c5e-7539-47d8-961e-928946fdcac6 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:08.817543 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927c9c5e-7539-47d8-961e-928946fdcac6 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:08.819010 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 436/879] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:34:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:34:08.825733 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96c9f19-f2ee-409d-a4f2-9f97914ed315 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:08.826363 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96c9f19-f2ee-409d-a4f2-9f97914ed315 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:08.826876 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96c9f19-f2ee-409d-a4f2-9f97914ed315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:08.847226 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-e96c9f19-f2ee-409d-a4f2-9f97914ed315 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:34:08.848836 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-e96c9f19-f2ee-409d-a4f2-9f97914ed315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:34:08.869382 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:08.940470 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 444/880] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:34:08 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 13:34:08.963954 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927c9c5e-7539-47d8-961e-928946fdcac6 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:09.083401 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:09.098321 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927c9c5e-7539-47d8-961e-928946fdcac6 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:09.135404 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf500a1c-456e-47de-bbd2-1795fc076b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:09.135957 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf500a1c-456e-47de-bbd2-1795fc076b4b None admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:09.136241 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf500a1c-456e-47de-bbd2-1795fc076b4b None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:09.136521 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf500a1c-456e-47de-bbd2-1795fc076b4b None admin] PATH_INFO: `/v3/services` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:09.137237 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf500a1c-456e-47de-bbd2-1795fc076b4b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:09.138765 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf500a1c-456e-47de-bbd2-1795fc076b4b None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:09.160298 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 437/881] 10.222.0.58 () {60 vars in 1141 bytes} [Tue Jul 26 13:34:08 2022] POST /identity/v3/services => generated 269 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:34:10.516685 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3350a7-66c1-480f-96bd-a853a4893cbf None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:10.517242 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3350a7-66c1-480f-96bd-a853a4893cbf None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:10.517242 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3350a7-66c1-480f-96bd-a853a4893cbf None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:10.518882 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 445/882] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:34:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:34:10.524202 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3098e0-fd5f-4fb0-9073-aa7d95455c33 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:10.524679 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3098e0-fd5f-4fb0-9073-aa7d95455c33 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:10.524769 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3098e0-fd5f-4fb0-9073-aa7d95455c33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:10.543387 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-2a3098e0-fd5f-4fb0-9073-aa7d95455c33 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:34:10.544587 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-2a3098e0-fd5f-4fb0-9073-aa7d95455c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:34:10.626726 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 438/883] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:34:10 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 13:34:10.649744 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3350a7-66c1-480f-96bd-a853a4893cbf None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:10.759062 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3350a7-66c1-480f-96bd-a853a4893cbf None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:10.795835 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb56a9d-369a-4845-a357-ca6b627bbc1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:10.796417 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb56a9d-369a-4845-a357-ca6b627bbc1f None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:10.796703 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb56a9d-369a-4845-a357-ca6b627bbc1f None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:10.797232 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb56a9d-369a-4845-a357-ca6b627bbc1f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:10.797919 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb56a9d-369a-4845-a357-ca6b627bbc1f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:10.799304 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb56a9d-369a-4845-a357-ca6b627bbc1f None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:10.809455 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-efb56a9d-369a-4845-a357-ca6b627bbc1f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 26 13:34:10.809455 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:10.809455 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 13:34:10.809455 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 13:34:10.809455 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 13:34:10.809455 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 13:34:10.809455 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 13:34:10.809455 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 13:34:10.809455 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 26 13:34:10.809455 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:34:10.809455 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 13:34:10.809455 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:34:10.809455 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:10.809455 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:34:10.809455 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:34:10.809455 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:34:10.809455 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:34:10.810509 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:34:10.810509 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:34:10.810509 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:34:10.810509 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:34:10.810509 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:34:10.810509 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:34:10.810509 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 13:34:10.810509 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 13:34:10.810509 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 13:34:10.810509 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 13:34:10.810509 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:34:10.810509 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:34:10.810509 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:34:10.810509 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:34:10.810509 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:34:10.810509 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:34:10.811331 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:34:10.811331 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:34:10.811331 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:34:10.811331 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:34:10.811331 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:34:10.811331 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:34:10.811331 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:34:10.811331 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:34:10.811331 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:34:10.811331 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:34:10.811331 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 13:34:10.811331 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 13:34:10.811331 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 26 13:34:10.811331 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:34:10.811331 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 446/884] 10.222.0.58 () {56 vars in 1114 bytes} [Tue Jul 26 13:34:10 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:34:10.816980 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3098e0-fd5f-4fb0-9073-aa7d95455c33 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:10.897818 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:10.931523 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3098e0-fd5f-4fb0-9073-aa7d95455c33 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:10.966739 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a325394-fa3e-4a46-a6de-956551aeb8a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:10.967366 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a325394-fa3e-4a46-a6de-956551aeb8a2 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:10.967650 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a325394-fa3e-4a46-a6de-956551aeb8a2 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:10.967924 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a325394-fa3e-4a46-a6de-956551aeb8a2 None admin] PATH_INFO: `/v3/services` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:10.968655 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a325394-fa3e-4a46-a6de-956551aeb8a2 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:10.968967 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a325394-fa3e-4a46-a6de-956551aeb8a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:10.970368 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a325394-fa3e-4a46-a6de-956551aeb8a2 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:10.975667 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 439/885] 10.222.0.58 () {56 vars in 1114 bytes} [Tue Jul 26 13:34:10 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:10.981527 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb56a9d-369a-4845-a357-ca6b627bbc1f None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:10.987632 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cca7a0-657e-454e-94c2-e98014985188 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:10.987918 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cca7a0-657e-454e-94c2-e98014985188 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:10.988196 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cca7a0-657e-454e-94c2-e98014985188 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:10.989784 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 440/886] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:34:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 13:34:10.995672 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f620441c-0fdc-4f86-8709-83f5d77e52ec None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:10.995961 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f620441c-0fdc-4f86-8709-83f5d77e52ec None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:10.996309 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f620441c-0fdc-4f86-8709-83f5d77e52ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:10.998182 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:11.016712 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-f620441c-0fdc-4f86-8709-83f5d77e52ec None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:34:11.017992 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-f620441c-0fdc-4f86-8709-83f5d77e52ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:34:11.105835 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb56a9d-369a-4845-a357-ca6b627bbc1f None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:11.106072 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 441/887] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:34:10 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 13:34:11.121793 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f620441c-0fdc-4f86-8709-83f5d77e52ec None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:11.140743 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfee54d4-7f18-416f-b8d1-3612ba565117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:11.141396 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfee54d4-7f18-416f-b8d1-3612ba565117 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:11.141724 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfee54d4-7f18-416f-b8d1-3612ba565117 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:11.142006 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfee54d4-7f18-416f-b8d1-3612ba565117 None admin] PATH_INFO: `/v3/services` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:11.142691 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfee54d4-7f18-416f-b8d1-3612ba565117 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:11.143000 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfee54d4-7f18-416f-b8d1-3612ba565117 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:11.144366 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfee54d4-7f18-416f-b8d1-3612ba565117 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:11.153811 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 447/888] 10.222.0.58 () {56 vars in 1114 bytes} [Tue Jul 26 13:34:10 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:11.155632 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfee54d4-7f18-416f-b8d1-3612ba565117 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:11.201941 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:11.240796 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f620441c-0fdc-4f86-8709-83f5d77e52ec None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:11.246909 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:11.271514 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfee54d4-7f18-416f-b8d1-3612ba565117 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:11.277899 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec4f475-26ed-4efb-8e9c-9f9d0e9ac868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:11.278645 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec4f475-26ed-4efb-8e9c-9f9d0e9ac868 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:11.279013 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec4f475-26ed-4efb-8e9c-9f9d0e9ac868 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:11.279354 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec4f475-26ed-4efb-8e9c-9f9d0e9ac868 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:11.290922 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec4f475-26ed-4efb-8e9c-9f9d0e9ac868 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:34:11.295791 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-aec4f475-26ed-4efb-8e9c-9f9d0e9ac868 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 13:34:11.295791 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:11.295791 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:34:11.295791 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:34:11.295791 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:34:11.295791 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:34:11.295791 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:34:11.295791 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:34:11.295791 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:34:11.295791 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:34:11.295791 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:34:11.295791 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:34:11.295791 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:34:11.295791 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:34:11.295791 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:34:11.295791 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:34:11.296771 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:34:11.296771 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:34:11.296771 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:34:11.296771 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:34:11.296771 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:34:11.296771 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:34:11.296771 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:34:11.296771 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:34:11.296771 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:34:11.296771 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:34:11.296771 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:34:11.296771 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:34:11.296771 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:34:11.296771 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:34:11.296771 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:34:11.296771 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:34:11.298586 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:34:11.298586 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:34:11.298586 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:34:11.298586 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:34:11.298586 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:34:11.298586 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:34:11.298586 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:34:11.298586 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:34:11.298586 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:34:11.298586 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:34:11.298586 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 13:34:11.298586 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:34:11.298586 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 442/889] 10.222.0.58 () {56 vars in 1120 bytes} [Tue Jul 26 13:34:11 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:34:11.299349 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa348138-2c93-458d-a664-b2ff1d544a95 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:11.299349 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa348138-2c93-458d-a664-b2ff1d544a95 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:11.299349 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa348138-2c93-458d-a664-b2ff1d544a95 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:11.300761 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 443/890] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:34:11 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:34:11.305525 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa348138-2c93-458d-a664-b2ff1d544a95 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:11.313507 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8a0f0d-4a9c-49e9-af68-a9b5e20da11e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:11.314139 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8a0f0d-4a9c-49e9-af68-a9b5e20da11e None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:11.314843 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8a0f0d-4a9c-49e9-af68-a9b5e20da11e None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:11.315126 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8a0f0d-4a9c-49e9-af68-a9b5e20da11e None admin] PATH_INFO: `/v3/endpoints` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:11.315884 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8a0f0d-4a9c-49e9-af68-a9b5e20da11e None admin] RBAC: Adding query filter params (interface=public, service_id=72cef16b60b54044a4d71c2ea7472c9d, region_id=RegionOne) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:11.316197 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8a0f0d-4a9c-49e9-af68-a9b5e20da11e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:11.318261 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8a0f0d-4a9c-49e9-af68-a9b5e20da11e None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:11.323659 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 448/891] 10.222.0.58 () {56 vars in 1251 bytes} [Tue Jul 26 13:34:11 2022] GET /identity/v3/endpoints?service_id=72cef16b60b54044a4d71c2ea7472c9d&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:34:11.326852 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c71f7ce-ac43-44b6-8490-7298669e14af None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:11.327378 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c71f7ce-ac43-44b6-8490-7298669e14af None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:11.328411 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c71f7ce-ac43-44b6-8490-7298669e14af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:11.346560 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-6c71f7ce-ac43-44b6-8490-7298669e14af None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:34:11.347760 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-6c71f7ce-ac43-44b6-8490-7298669e14af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:34:11.412813 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa348138-2c93-458d-a664-b2ff1d544a95 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:11.414427 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 449/892] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:34:11 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 13:34:11.415278 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c71f7ce-ac43-44b6-8490-7298669e14af None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:11.440148 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:11.448195 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3651233e-c03e-4d89-81c1-ca4ae4e85a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:11.448817 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3651233e-c03e-4d89-81c1-ca4ae4e85a50 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:11.449102 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3651233e-c03e-4d89-81c1-ca4ae4e85a50 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:11.449379 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3651233e-c03e-4d89-81c1-ca4ae4e85a50 None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:11.450131 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3651233e-c03e-4d89-81c1-ca4ae4e85a50 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:11.450443 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3651233e-c03e-4d89-81c1-ca4ae4e85a50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:11.451805 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3651233e-c03e-4d89-81c1-ca4ae4e85a50 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:11.460137 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 444/893] 10.222.0.58 () {56 vars in 1115 bytes} [Tue Jul 26 13:34:11 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:11.461864 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f80b9f-ec6e-4526-acf1-f0ff3f90a2a5 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:11.462169 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f80b9f-ec6e-4526-acf1-f0ff3f90a2a5 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:11.462430 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f80b9f-ec6e-4526-acf1-f0ff3f90a2a5 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:11.463640 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 445/894] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:34:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:34:11.464319 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f80b9f-ec6e-4526-acf1-f0ff3f90a2a5 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:11.494229 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:11.526527 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c71f7ce-ac43-44b6-8490-7298669e14af None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:11.564140 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad75522e-bd48-415e-9a39-fece0416bfde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:11.564819 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad75522e-bd48-415e-9a39-fece0416bfde None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:11.565099 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad75522e-bd48-415e-9a39-fece0416bfde None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:11.565372 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad75522e-bd48-415e-9a39-fece0416bfde None admin] PATH_INFO: `/v3/services` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:11.566063 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad75522e-bd48-415e-9a39-fece0416bfde None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:11.567523 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad75522e-bd48-415e-9a39-fece0416bfde None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:11.574234 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 450/895] 10.222.0.58 () {56 vars in 1087 bytes} [Tue Jul 26 13:34:11 2022] GET /identity/v3/services => generated 1268 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:34:11.574835 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad75522e-bd48-415e-9a39-fece0416bfde None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:11.576606 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f80b9f-ec6e-4526-acf1-f0ff3f90a2a5 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:11.635275 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc9bd12-72e3-4d25-abb9-60cf60c23d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:11.635922 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc9bd12-72e3-4d25-abb9-60cf60c23d8b None admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:11.636204 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc9bd12-72e3-4d25-abb9-60cf60c23d8b None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:11.636478 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc9bd12-72e3-4d25-abb9-60cf60c23d8b None admin] PATH_INFO: `/v3/services` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:11.637213 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc9bd12-72e3-4d25-abb9-60cf60c23d8b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:11.638762 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc9bd12-72e3-4d25-abb9-60cf60c23d8b None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:11.656826 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 446/896] 10.222.0.58 () {60 vars in 1141 bytes} [Tue Jul 26 13:34:11 2022] POST /identity/v3/services => generated 241 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:34:11.658268 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dd77d5-285e-4b48-b8db-7acb9ebd0c65 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:11.658613 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dd77d5-285e-4b48-b8db-7acb9ebd0c65 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:11.658914 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dd77d5-285e-4b48-b8db-7acb9ebd0c65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:11.683310 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-b1dd77d5-285e-4b48-b8db-7acb9ebd0c65 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:34:11.684547 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-b1dd77d5-285e-4b48-b8db-7acb9ebd0c65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:34:11.702340 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad75522e-bd48-415e-9a39-fece0416bfde None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:11.748189 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:11.749528 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:11.756374 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ae4cec-f96f-476d-97c0-5f026def7ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:11.756374 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ae4cec-f96f-476d-97c0-5f026def7ada None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:11.756374 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ae4cec-f96f-476d-97c0-5f026def7ada None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:11.756374 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ae4cec-f96f-476d-97c0-5f026def7ada None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:11.780649 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ae4cec-f96f-476d-97c0-5f026def7ada None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:11.781882 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ae4cec-f96f-476d-97c0-5f026def7ada None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:11.799306 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 447/897] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:34:11 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 13:34:11.801325 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-c7ae4cec-f96f-476d-97c0-5f026def7ada None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:34:11.801325 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:11.801325 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:34:11.801325 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:34:11.801325 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:34:11.801325 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:34:11.801325 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:34:11.801325 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:34:11.801325 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:34:11.801325 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:34:11.801325 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:34:11.801325 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:34:11.801325 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 13:34:11.801325 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 13:34:11.801325 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 13:34:11.802507 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 13:34:11.802507 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:34:11.802507 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:34:11.802507 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:34:11.802507 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:34:11.802507 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:34:11.802507 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:34:11.802507 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:34:11.802507 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:34:11.802507 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:34:11.802507 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:34:11.802507 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:34:11.802507 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:34:11.802507 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:34:11.802507 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:34:11.802507 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:34:11.802507 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:34:11.803399 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 13:34:11.803399 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 13:34:11.803399 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:34:11.803399 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:34:11.803399 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 451/898] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:34:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:34:11.808459 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dd77d5-285e-4b48-b8db-7acb9ebd0c65 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:11.827933 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ae4cec-f96f-476d-97c0-5f026def7ada None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:11.956642 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dd77d5-285e-4b48-b8db-7acb9ebd0c65 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:11.965652 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ae4cec-f96f-476d-97c0-5f026def7ada None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:12.002477 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095df6fb-49bf-4275-818a-f15d8b47fd52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:12.003236 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095df6fb-49bf-4275-818a-f15d8b47fd52 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:12.003523 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095df6fb-49bf-4275-818a-f15d8b47fd52 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:12.003800 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095df6fb-49bf-4275-818a-f15d8b47fd52 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:12.003921 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a206f8c-2d1f-4b3e-bd07-826fb7bd36d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:12.004447 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095df6fb-49bf-4275-818a-f15d8b47fd52 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:12.004606 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a206f8c-2d1f-4b3e-bd07-826fb7bd36d1 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:12.005003 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a206f8c-2d1f-4b3e-bd07-826fb7bd36d1 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:12.005347 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a206f8c-2d1f-4b3e-bd07-826fb7bd36d1 None admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:12.005861 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095df6fb-49bf-4275-818a-f15d8b47fd52 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:12.006212 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a206f8c-2d1f-4b3e-bd07-826fb7bd36d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:12.006767 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a206f8c-2d1f-4b3e-bd07-826fb7bd36d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:12.009141 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a206f8c-2d1f-4b3e-bd07-826fb7bd36d1 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:12.016748 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-095df6fb-49bf-4275-818a-f15d8b47fd52 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 26 13:34:12.016748 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:12.016748 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 13:34:12.016748 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 13:34:12.016748 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 13:34:12.016748 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 13:34:12.016748 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 13:34:12.016748 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 13:34:12.016748 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 26 13:34:12.016748 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:34:12.016748 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 13:34:12.016748 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:34:12.016748 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:12.016748 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:34:12.016748 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:34:12.016748 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:34:12.017953 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:34:12.017953 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:34:12.017953 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:34:12.017953 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:34:12.017953 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:34:12.017953 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:34:12.017953 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:34:12.017953 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 13:34:12.017953 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 13:34:12.017953 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 13:34:12.017953 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 13:34:12.017953 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:34:12.017953 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:34:12.017953 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:34:12.017953 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:34:12.017953 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:34:12.018786 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:34:12.018786 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:34:12.018786 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:34:12.018786 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:34:12.018786 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:34:12.018786 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:34:12.018786 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:34:12.018786 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:34:12.018786 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:34:12.018786 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:34:12.018786 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:34:12.018786 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 13:34:12.018786 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 13:34:12.018786 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 26 13:34:12.018786 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:34:12.023883 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 452/899] 10.222.0.58 () {56 vars in 1132 bytes} [Tue Jul 26 13:34:11 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:34:12.024953 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 448/900] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Jul 26 13:34:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:12.025571 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095df6fb-49bf-4275-818a-f15d8b47fd52 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:12.030692 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a206f8c-2d1f-4b3e-bd07-826fb7bd36d1 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:12.096113 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:12.098479 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:12.167128 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a206f8c-2d1f-4b3e-bd07-826fb7bd36d1 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:12.176282 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095df6fb-49bf-4275-818a-f15d8b47fd52 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:12.210205 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec581602-c7e0-4b5f-b934-1deaa54d4ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:12.211243 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec581602-c7e0-4b5f-b934-1deaa54d4ad3 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:12.211759 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec581602-c7e0-4b5f-b934-1deaa54d4ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:12.212253 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec581602-c7e0-4b5f-b934-1deaa54d4ad3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:12.221235 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec581602-c7e0-4b5f-b934-1deaa54d4ad3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:12.223290 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7413f9b-acd4-4244-b1a7-dd703caa6726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:12.223543 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec581602-c7e0-4b5f-b934-1deaa54d4ad3 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:12.223896 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7413f9b-acd4-4244-b1a7-dd703caa6726 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:12.224181 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7413f9b-acd4-4244-b1a7-dd703caa6726 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:12.224526 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7413f9b-acd4-4244-b1a7-dd703caa6726 None admin] PATH_INFO: `/v3/services` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:12.225273 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7413f9b-acd4-4244-b1a7-dd703caa6726 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:12.225622 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7413f9b-acd4-4244-b1a7-dd703caa6726 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:12.228075 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7413f9b-acd4-4244-b1a7-dd703caa6726 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:12.234035 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 453/901] 10.222.0.58 () {56 vars in 1126 bytes} [Tue Jul 26 13:34:12 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:34:12.241502 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-ec581602-c7e0-4b5f-b934-1deaa54d4ad3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 13:34:12.241502 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:12.241502 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:34:12.241502 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:34:12.241502 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:34:12.241502 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:34:12.241502 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:34:12.241502 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:34:12.241502 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:34:12.241502 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:34:12.241502 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:34:12.241502 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:34:12.241502 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 13:34:12.241502 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 13:34:12.241502 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 13:34:12.241502 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 13:34:12.243482 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:34:12.243482 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:34:12.243482 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 13:34:12.243482 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 13:34:12.243482 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 13:34:12.243482 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 13:34:12.243482 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:34:12.243482 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:34:12.243482 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:34:12.243482 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:34:12.243482 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:34:12.243482 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:34:12.243482 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:34:12.243482 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:34:12.243482 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:34:12.243482 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:34:12.243482 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:34:12.243482 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:34:12.244983 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:34:12.244983 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:34:12.244983 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 13:34:12.244983 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 13:34:12.244983 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 13:34:12.244983 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 13:34:12.244983 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 13:34:12.244983 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 13:34:12.244983 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 13:34:12.244983 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:34:12.247425 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7413f9b-acd4-4244-b1a7-dd703caa6726 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:12.248440 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 449/902] 10.222.0.58 () {56 vars in 1117 bytes} [Tue Jul 26 13:34:12 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 13:34:12.249708 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec581602-c7e0-4b5f-b934-1deaa54d4ad3 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:12.361827 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:12.365904 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec581602-c7e0-4b5f-b934-1deaa54d4ad3 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:12.400557 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7413f9b-acd4-4244-b1a7-dd703caa6726 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:12.404216 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c6eca3-c277-408f-b2fd-5a583c78992f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:12.404568 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c6eca3-c277-408f-b2fd-5a583c78992f None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:12.404949 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c6eca3-c277-408f-b2fd-5a583c78992f None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:12.405320 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c6eca3-c277-408f-b2fd-5a583c78992f None admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:12.406256 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c6eca3-c277-408f-b2fd-5a583c78992f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:12.406663 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c6eca3-c277-408f-b2fd-5a583c78992f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:12.408194 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c6eca3-c277-408f-b2fd-5a583c78992f None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:12.415423 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:12.434836 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 450/903] 10.222.0.58 () {56 vars in 1149 bytes} [Tue Jul 26 13:34:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:12.436289 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c6eca3-c277-408f-b2fd-5a583c78992f None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:12.449877 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b4ed28-2415-480d-b537-b59ef7f53ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:12.450607 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b4ed28-2415-480d-b537-b59ef7f53ea2 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:12.450892 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b4ed28-2415-480d-b537-b59ef7f53ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:12.451171 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b4ed28-2415-480d-b537-b59ef7f53ea2 None admin] PATH_INFO: `/v3/services` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:12.451857 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b4ed28-2415-480d-b537-b59ef7f53ea2 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:12.452161 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b4ed28-2415-480d-b537-b59ef7f53ea2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:12.453568 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b4ed28-2415-480d-b537-b59ef7f53ea2 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:12.464209 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 454/904] 10.222.0.58 () {56 vars in 1126 bytes} [Tue Jul 26 13:34:12 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:34:12.465462 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b4ed28-2415-480d-b537-b59ef7f53ea2 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:12.568611 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c6eca3-c277-408f-b2fd-5a583c78992f None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:12.606241 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb0ea1a-835f-4d22-aadb-d3d0e1028d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:12.606961 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb0ea1a-835f-4d22-aadb-d3d0e1028d1e None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:12.607355 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb0ea1a-835f-4d22-aadb-d3d0e1028d1e None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:12.607739 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb0ea1a-835f-4d22-aadb-d3d0e1028d1e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:12.613323 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b4ed28-2415-480d-b537-b59ef7f53ea2 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:12.643687 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:12.668175 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c545f5-6dd3-4a84-8f4c-47016a142596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:12.668921 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c545f5-6dd3-4a84-8f4c-47016a142596 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:12.669434 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c545f5-6dd3-4a84-8f4c-47016a142596 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:12.669795 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c545f5-6dd3-4a84-8f4c-47016a142596 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:12.670981 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c545f5-6dd3-4a84-8f4c-47016a142596 None admin] RBAC: Adding query filter params (interface=public, service_id=3bfa3474636f4661901f632d77a8baec, region_id=RegionOne) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:12.671096 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c545f5-6dd3-4a84-8f4c-47016a142596 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:12.672812 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c545f5-6dd3-4a84-8f4c-47016a142596 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:12.683828 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 455/905] 10.222.0.58 () {56 vars in 1251 bytes} [Tue Jul 26 13:34:12 2022] GET /identity/v3/endpoints?service_id=3bfa3474636f4661901f632d77a8baec&interface=public®ion_id=RegionOne => generated 194 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:12.685376 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c545f5-6dd3-4a84-8f4c-47016a142596 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:12.698694 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb0ea1a-835f-4d22-aadb-d3d0e1028d1e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:12.701169 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb0ea1a-835f-4d22-aadb-d3d0e1028d1e None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:12.732447 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:12.814861 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c545f5-6dd3-4a84-8f4c-47016a142596 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:12.828583 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 451/906] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Jul 26 13:34:12 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 13:34:12.834821 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb0ea1a-835f-4d22-aadb-d3d0e1028d1e None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:12.861999 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f4f524-6a6b-4fb3-b5d6-a78b2fde97b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:12.863071 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f4f524-6a6b-4fb3-b5d6-a78b2fde97b1 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:12.863638 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f4f524-6a6b-4fb3-b5d6-a78b2fde97b1 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:12.863921 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f4f524-6a6b-4fb3-b5d6-a78b2fde97b1 None admin] PATH_INFO: `/v3/services` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:12.864835 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f4f524-6a6b-4fb3-b5d6-a78b2fde97b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:12.866968 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f4f524-6a6b-4fb3-b5d6-a78b2fde97b1 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:12.874863 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 456/907] 10.222.0.58 () {56 vars in 1087 bytes} [Tue Jul 26 13:34:12 2022] GET /identity/v3/services => generated 1497 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:34:12.910051 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:12.953363 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb0ea1a-835f-4d22-aadb-d3d0e1028d1e None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:12.991898 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c952bec3-7854-4820-99e9-acfdc45d44b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:12.992484 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c952bec3-7854-4820-99e9-acfdc45d44b3 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:12.992770 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c952bec3-7854-4820-99e9-acfdc45d44b3 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:12.993051 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c952bec3-7854-4820-99e9-acfdc45d44b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:13.004595 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c952bec3-7854-4820-99e9-acfdc45d44b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:13.006106 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c952bec3-7854-4820-99e9-acfdc45d44b3 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:13.022329 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-c952bec3-7854-4820-99e9-acfdc45d44b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:34:13.022329 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:13.022329 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:34:13.022329 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:34:13.022329 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:34:13.022329 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:34:13.022329 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:34:13.022329 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:34:13.022329 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:34:13.022329 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:34:13.022329 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:34:13.022329 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:34:13.022329 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 13:34:13.022329 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 13:34:13.022329 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 13:34:13.024023 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 13:34:13.024023 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:34:13.024023 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:34:13.024023 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:34:13.024023 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:34:13.024023 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:34:13.024023 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:34:13.024023 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:34:13.024023 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:34:13.024023 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:34:13.024023 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:34:13.024023 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:34:13.024023 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:34:13.024023 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:34:13.024023 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:34:13.024023 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:34:13.024023 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:34:13.025206 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 13:34:13.025206 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 13:34:13.025206 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:34:13.025206 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:34:13.025206 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 452/908] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:34:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 13:34:13.037286 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f4f524-6a6b-4fb3-b5d6-a78b2fde97b1 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:13.156746 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f4f524-6a6b-4fb3-b5d6-a78b2fde97b1 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:13.158830 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:13.196351 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f46d1bb-bf28-4c43-bc14-3b741ff00449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:13.196988 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f46d1bb-bf28-4c43-bc14-3b741ff00449 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:13.197272 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f46d1bb-bf28-4c43-bc14-3b741ff00449 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:13.197578 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f46d1bb-bf28-4c43-bc14-3b741ff00449 None admin] PATH_INFO: `/v3/domains` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:13.198280 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f46d1bb-bf28-4c43-bc14-3b741ff00449 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:13.198586 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f46d1bb-bf28-4c43-bc14-3b741ff00449 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:13.199947 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f46d1bb-bf28-4c43-bc14-3b741ff00449 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:13.217617 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 457/909] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Jul 26 13:34:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:13.220316 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c952bec3-7854-4820-99e9-acfdc45d44b3 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:13.334251 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c952bec3-7854-4820-99e9-acfdc45d44b3 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:13.346757 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:13.374280 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8bb0f5-8080-412f-bc84-cecdb3dc14c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:13.375304 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8bb0f5-8080-412f-bc84-cecdb3dc14c3 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:13.375766 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8bb0f5-8080-412f-bc84-cecdb3dc14c3 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:13.376221 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8bb0f5-8080-412f-bc84-cecdb3dc14c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:13.384390 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8bb0f5-8080-412f-bc84-cecdb3dc14c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:13.386231 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8bb0f5-8080-412f-bc84-cecdb3dc14c3 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:13.399463 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-fb8bb0f5-8080-412f-bc84-cecdb3dc14c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 13:34:13.399463 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:13.399463 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:34:13.399463 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:34:13.399463 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:34:13.399463 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:34:13.399463 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:34:13.399463 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:34:13.399463 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:34:13.399463 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:34:13.399463 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:34:13.399463 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:34:13.399463 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 13:34:13.399463 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 13:34:13.399463 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 13:34:13.400420 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 13:34:13.400420 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:34:13.400420 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:34:13.400420 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:34:13.400420 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:34:13.400420 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:34:13.400420 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:34:13.400420 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:34:13.400420 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:34:13.400420 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:34:13.400420 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:34:13.400420 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:34:13.400420 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:34:13.400420 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:34:13.400420 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:34:13.400420 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:34:13.400420 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:34:13.401314 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 13:34:13.401314 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 13:34:13.401314 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 13:34:13.401314 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 13:34:13.401314 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 13:34:13.401314 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 13:34:13.401314 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 13:34:13.401314 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:34:13.408204 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 453/910] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Jul 26 13:34:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 13:34:13.409798 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f46d1bb-bf28-4c43-bc14-3b741ff00449 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:13.529536 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f46d1bb-bf28-4c43-bc14-3b741ff00449 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:13.566855 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fd7d80-e810-4951-99fa-8c68d3c52aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:13.567735 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fd7d80-e810-4951-99fa-8c68d3c52aab None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:13.568204 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fd7d80-e810-4951-99fa-8c68d3c52aab None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:13.568634 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fd7d80-e810-4951-99fa-8c68d3c52aab None admin] PATH_INFO: `/v3/projects` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:13.569574 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fd7d80-e810-4951-99fa-8c68d3c52aab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:13.570641 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fd7d80-e810-4951-99fa-8c68d3c52aab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:13.572220 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fd7d80-e810-4951-99fa-8c68d3c52aab None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:13.589682 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 458/911] 10.222.0.58 () {56 vars in 1148 bytes} [Tue Jul 26 13:34:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:13.592454 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8bb0f5-8080-412f-bc84-cecdb3dc14c3 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:13.677096 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1ba203-530f-4384-b180-5fcbc7069482 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:13.677650 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1ba203-530f-4384-b180-5fcbc7069482 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:13.677650 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1ba203-530f-4384-b180-5fcbc7069482 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:13.680300 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 459/912] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:34:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 13:34:13.686489 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cd0834-1d63-4e2b-aa40-adc806344971 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:13.686940 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cd0834-1d63-4e2b-aa40-adc806344971 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:13.687335 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cd0834-1d63-4e2b-aa40-adc806344971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:13.706417 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8bb0f5-8080-412f-bc84-cecdb3dc14c3 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:13.708640 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-45cd0834-1d63-4e2b-aa40-adc806344971 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:34:13.710424 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-45cd0834-1d63-4e2b-aa40-adc806344971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:34:13.729096 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:13.745730 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6d2923-4848-432e-a9a2-b138b59b954a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:13.746338 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6d2923-4848-432e-a9a2-b138b59b954a None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:13.746600 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6d2923-4848-432e-a9a2-b138b59b954a None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:13.746876 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6d2923-4848-432e-a9a2-b138b59b954a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:13.747684 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6d2923-4848-432e-a9a2-b138b59b954a None admin] RBAC: Adding query filter params (role.id=ac11ff5f40194d7d98f81a5429aa3826, scope.project.id=1e8bb4385c204210a1bc77b1f0d654f3, user.id=b64bd7b183d24d8c9037b261c7884810) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:13.747997 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6d2923-4848-432e-a9a2-b138b59b954a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:13.750138 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6d2923-4848-432e-a9a2-b138b59b954a None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:13.765827 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 454/913] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Jul 26 13:34:13 2022] GET /identity/v3/role_assignments?user.id=b64bd7b183d24d8c9037b261c7884810&scope.project.id=1e8bb4385c204210a1bc77b1f0d654f3&role.id=ac11ff5f40194d7d98f81a5429aa3826 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:13.801691 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 460/914] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:34:13 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 13:34:13.820598 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6d2923-4848-432e-a9a2-b138b59b954a None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:13.846156 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745cb27b-d3f4-42e7-8c13-2fd3bb302631 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:13.846751 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745cb27b-d3f4-42e7-8c13-2fd3bb302631 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:13.847134 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745cb27b-d3f4-42e7-8c13-2fd3bb302631 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:13.849005 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 461/915] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:34:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 13:34:13.855910 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6f6d59-6365-4220-a0a0-151ff65b245b None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:13.856353 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6f6d59-6365-4220-a0a0-151ff65b245b None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:13.856730 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6f6d59-6365-4220-a0a0-151ff65b245b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:13.865742 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:13.876070 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-ca6f6d59-6365-4220-a0a0-151ff65b245b None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:34:13.877457 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-ca6f6d59-6365-4220-a0a0-151ff65b245b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:34:13.949223 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6d2923-4848-432e-a9a2-b138b59b954a None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:13.970736 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 462/916] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:34:13 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 13:34:13.985578 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cc0649-8918-4d66-b118-703138434e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:13.986443 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cc0649-8918-4d66-b118-703138434e56 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:13.986881 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cc0649-8918-4d66-b118-703138434e56 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:13.987308 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cc0649-8918-4d66-b118-703138434e56 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:13.988088 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cc0649-8918-4d66-b118-703138434e56 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:13.989643 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cc0649-8918-4d66-b118-703138434e56 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:13.989821 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6f6d59-6365-4220-a0a0-151ff65b245b None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:13.999665 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-67cc0649-8918-4d66-b118-703138434e56 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 26 13:34:13.999665 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:13.999665 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 13:34:13.999665 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 13:34:13.999665 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 13:34:13.999665 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 13:34:13.999665 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 13:34:13.999665 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 13:34:13.999665 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 26 13:34:13.999665 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:34:13.999665 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 13:34:13.999665 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:34:13.999665 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:13.999665 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:34:13.999665 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:34:13.999665 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:34:13.999665 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:34:14.000743 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:34:14.000743 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:34:14.000743 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:34:14.000743 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:34:14.000743 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:34:14.000743 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:34:14.000743 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 13:34:14.000743 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 13:34:14.000743 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 13:34:14.000743 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 13:34:14.000743 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:34:14.000743 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:34:14.000743 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:34:14.000743 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:34:14.000743 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:34:14.000743 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:34:14.001592 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:34:14.001592 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:34:14.001592 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:34:14.001592 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:34:14.001592 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:34:14.001592 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:34:14.001592 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:34:14.001592 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:34:14.001592 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:34:14.001592 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:34:14.001592 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 13:34:14.001592 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 13:34:14.001592 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 26 13:34:14.001592 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:34:14.002754 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 455/917] 10.222.0.58 () {56 vars in 1114 bytes} [Tue Jul 26 13:34:13 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:34:14.008658 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cc0649-8918-4d66-b118-703138434e56 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:14.090000 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:14.116667 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6f6d59-6365-4220-a0a0-151ff65b245b None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:14.121388 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:14.125748 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cc0649-8918-4d66-b118-703138434e56 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:14.165021 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c03ec44-0f27-4968-bd81-17043c726bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:14.165784 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c03ec44-0f27-4968-bd81-17043c726bff None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:14.166076 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c03ec44-0f27-4968-bd81-17043c726bff None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:14.166352 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c03ec44-0f27-4968-bd81-17043c726bff None admin] PATH_INFO: `/v3/services` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:14.166991 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6cc765-8ff4-4f05-bde2-1a41ab908fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:14.167128 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c03ec44-0f27-4968-bd81-17043c726bff None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:14.167334 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c03ec44-0f27-4968-bd81-17043c726bff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:14.168174 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6cc765-8ff4-4f05-bde2-1a41ab908fad None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:14.168653 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6cc765-8ff4-4f05-bde2-1a41ab908fad None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:14.168768 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c03ec44-0f27-4968-bd81-17043c726bff None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:14.169236 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6cc765-8ff4-4f05-bde2-1a41ab908fad None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:14.170423 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6cc765-8ff4-4f05-bde2-1a41ab908fad None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:14.172962 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6cc765-8ff4-4f05-bde2-1a41ab908fad None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:14.174871 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 456/918] 10.222.0.58 () {56 vars in 1114 bytes} [Tue Jul 26 13:34:14 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:34:14.181082 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c03ec44-0f27-4968-bd81-17043c726bff None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:14.188564 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-9e6cc765-8ff4-4f05-bde2-1a41ab908fad None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 13:34:14.188564 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:14.188564 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 13:34:14.188564 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 13:34:14.188564 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 13:34:14.188564 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 13:34:14.188564 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 13:34:14.188564 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 13:34:14.188564 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 13:34:14.188564 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:34:14.188564 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 13:34:14.188564 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:34:14.188564 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:14.188564 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:34:14.188564 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:34:14.188564 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:34:14.188564 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:34:14.189605 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:34:14.189605 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:34:14.189605 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:34:14.189605 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:34:14.189605 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:34:14.189605 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:34:14.189605 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 13:34:14.189605 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 13:34:14.189605 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 13:34:14.189605 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 13:34:14.189605 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:34:14.189605 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:34:14.189605 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:34:14.189605 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:34:14.189605 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:34:14.189605 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:34:14.190443 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:34:14.190443 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:34:14.190443 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:34:14.190443 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:34:14.190443 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:34:14.190443 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:34:14.190443 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:34:14.190443 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:34:14.190443 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:34:14.190443 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:34:14.190443 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 13:34:14.190443 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 13:34:14.190443 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 13:34:14.190443 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:34:14.190443 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 463/919] 10.222.0.58 () {56 vars in 1126 bytes} [Tue Jul 26 13:34:13 2022] GET /identity/v3/services/object-store => generated 93 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:34:14.199249 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6cc765-8ff4-4f05-bde2-1a41ab908fad None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:14.353315 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c03ec44-0f27-4968-bd81-17043c726bff None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:14.365598 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6cc765-8ff4-4f05-bde2-1a41ab908fad None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:14.393187 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e881b280-b4d7-4e5a-80fe-185dae309cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:14.394132 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e881b280-b4d7-4e5a-80fe-185dae309cbe None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:14.394619 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e881b280-b4d7-4e5a-80fe-185dae309cbe None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:14.395071 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e881b280-b4d7-4e5a-80fe-185dae309cbe None admin] PATH_INFO: `/v3/services` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:14.395928 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e881b280-b4d7-4e5a-80fe-185dae309cbe None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:14.396422 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e881b280-b4d7-4e5a-80fe-185dae309cbe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:14.398519 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e881b280-b4d7-4e5a-80fe-185dae309cbe None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:14.408173 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98764722-d4d4-4be4-a140-d7077bcecfb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:14.408173 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98764722-d4d4-4be4-a140-d7077bcecfb5 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:14.408173 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98764722-d4d4-4be4-a140-d7077bcecfb5 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:14.408478 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98764722-d4d4-4be4-a140-d7077bcecfb5 None admin] PATH_INFO: `/v3/services` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:14.408922 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 457/920] 10.222.0.58 () {56 vars in 1114 bytes} [Tue Jul 26 13:34:14 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:34:14.409162 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98764722-d4d4-4be4-a140-d7077bcecfb5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:14.409490 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98764722-d4d4-4be4-a140-d7077bcecfb5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:14.410231 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e881b280-b4d7-4e5a-80fe-185dae309cbe None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:14.410837 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98764722-d4d4-4be4-a140-d7077bcecfb5 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:14.421544 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 464/921] 10.222.0.58 () {56 vars in 1122 bytes} [Tue Jul 26 13:34:14 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:14.425057 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98764722-d4d4-4be4-a140-d7077bcecfb5 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:14.441920 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:14.443403 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:14.525462 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e881b280-b4d7-4e5a-80fe-185dae309cbe None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:14.555095 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98764722-d4d4-4be4-a140-d7077bcecfb5 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:14.575926 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3cf3a9-c939-4d0e-abf6-80e2e012c474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:14.576558 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3cf3a9-c939-4d0e-abf6-80e2e012c474 None admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:14.576838 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3cf3a9-c939-4d0e-abf6-80e2e012c474 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:14.577108 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3cf3a9-c939-4d0e-abf6-80e2e012c474 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:14.577863 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3cf3a9-c939-4d0e-abf6-80e2e012c474 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:14.579289 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3cf3a9-c939-4d0e-abf6-80e2e012c474 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:14.600531 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f71f074-aaa0-4765-b377-373b405b1fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:14.601626 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f71f074-aaa0-4765-b377-373b405b1fb8 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:14.602289 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f71f074-aaa0-4765-b377-373b405b1fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:14.602869 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f71f074-aaa0-4765-b377-373b405b1fb8 None admin] PATH_INFO: `/v3/services` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:14.603778 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f71f074-aaa0-4765-b377-373b405b1fb8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:14.604462 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f71f074-aaa0-4765-b377-373b405b1fb8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:14.606753 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f71f074-aaa0-4765-b377-373b405b1fb8 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:14.618047 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 465/923] 10.222.0.58 () {56 vars in 1122 bytes} [Tue Jul 26 13:34:14 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:34:14.618240 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 458/923] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 13:34:14 2022] POST /identity/v3/endpoints => generated 346 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:34:14.619816 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3cf3a9-c939-4d0e-abf6-80e2e012c474 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:14.724282 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:14.744730 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3cf3a9-c939-4d0e-abf6-80e2e012c474 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:14.784455 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf5254a-a9c8-4eba-b7a2-6dd78d24ad93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:14.785377 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf5254a-a9c8-4eba-b7a2-6dd78d24ad93 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:14.785815 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf5254a-a9c8-4eba-b7a2-6dd78d24ad93 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:14.786298 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf5254a-a9c8-4eba-b7a2-6dd78d24ad93 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:14.787254 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf5254a-a9c8-4eba-b7a2-6dd78d24ad93 None admin] RBAC: Adding query filter params (interface=public, service_id=9fe7cfa7b96b411f8683612376ff5669, region_id=RegionOne) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:14.787704 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf5254a-a9c8-4eba-b7a2-6dd78d24ad93 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:14.789236 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf5254a-a9c8-4eba-b7a2-6dd78d24ad93 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:14.799836 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 459/924] 10.222.0.58 () {56 vars in 1251 bytes} [Tue Jul 26 13:34:14 2022] GET /identity/v3/endpoints?service_id=9fe7cfa7b96b411f8683612376ff5669&interface=public®ion_id=RegionOne => generated 194 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:34:14.801516 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f71f074-aaa0-4765-b377-373b405b1fb8 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:14.881837 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:14.913728 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f71f074-aaa0-4765-b377-373b405b1fb8 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:14.966476 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df59d67-0040-46b2-bfd2-b4f604419b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:14.967631 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df59d67-0040-46b2-bfd2-b4f604419b75 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:14.968194 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df59d67-0040-46b2-bfd2-b4f604419b75 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:14.968745 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df59d67-0040-46b2-bfd2-b4f604419b75 None admin] PATH_INFO: `/v3/services` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:14.969628 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df59d67-0040-46b2-bfd2-b4f604419b75 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:14.971316 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df59d67-0040-46b2-bfd2-b4f604419b75 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:14.983979 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 466/925] 10.222.0.58 () {56 vars in 1087 bytes} [Tue Jul 26 13:34:14 2022] GET /identity/v3/services => generated 1497 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:34:15.171139 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0c8ad3-c6dd-45ec-946d-caaac04e136a None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:15.171550 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0c8ad3-c6dd-45ec-946d-caaac04e136a None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:15.171983 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0c8ad3-c6dd-45ec-946d-caaac04e136a None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:15.173630 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 460/926] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:34:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:34:15.180076 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779735c3-826d-4c15-82c5-9137d2f4f4c1 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:15.187152 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779735c3-826d-4c15-82c5-9137d2f4f4c1 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:15.187152 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779735c3-826d-4c15-82c5-9137d2f4f4c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:15.201909 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-779735c3-826d-4c15-82c5-9137d2f4f4c1 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:34:15.204198 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-779735c3-826d-4c15-82c5-9137d2f4f4c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:34:15.286765 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 467/927] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:34:15 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 13:34:15.313263 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0c8ad3-c6dd-45ec-946d-caaac04e136a None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:15.492600 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0c8ad3-c6dd-45ec-946d-caaac04e136a None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:15.554570 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0038fbd-d334-4c5e-bac6-e6f4293ca354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:15.556004 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0038fbd-d334-4c5e-bac6-e6f4293ca354 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:15.556487 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0038fbd-d334-4c5e-bac6-e6f4293ca354 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:15.556964 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0038fbd-d334-4c5e-bac6-e6f4293ca354 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:15.558072 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0038fbd-d334-4c5e-bac6-e6f4293ca354 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:15.560559 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0038fbd-d334-4c5e-bac6-e6f4293ca354 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:15.577351 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-a0038fbd-d334-4c5e-bac6-e6f4293ca354 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 26 13:34:15.577351 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:15.577351 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 13:34:15.577351 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 13:34:15.577351 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 13:34:15.577351 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 13:34:15.577351 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 13:34:15.577351 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 13:34:15.577351 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 26 13:34:15.577351 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:34:15.577351 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 13:34:15.577351 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:34:15.577351 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:15.577351 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:34:15.577351 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:34:15.577351 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:34:15.579180 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:34:15.579180 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:34:15.579180 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:34:15.579180 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:34:15.579180 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:34:15.579180 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:34:15.579180 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:34:15.579180 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 13:34:15.579180 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 13:34:15.579180 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 13:34:15.579180 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 13:34:15.579180 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:34:15.579180 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:34:15.579180 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:34:15.579180 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:34:15.579180 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:34:15.580796 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:34:15.580796 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:34:15.580796 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:34:15.580796 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:34:15.580796 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:34:15.580796 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:34:15.580796 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:34:15.580796 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:34:15.580796 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:34:15.580796 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:34:15.580796 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:34:15.580796 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 13:34:15.580796 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 13:34:15.580796 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 26 13:34:15.580796 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:34:15.580796 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 461/928] 10.222.0.58 () {56 vars in 1132 bytes} [Tue Jul 26 13:34:15 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:34:15.594121 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779735c3-826d-4c15-82c5-9137d2f4f4c1 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:15.689769 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:15.729488 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779735c3-826d-4c15-82c5-9137d2f4f4c1 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:15.766662 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b82a3ca-360c-4bc9-9231-f07821d4bf89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:15.767668 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b82a3ca-360c-4bc9-9231-f07821d4bf89 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:15.768180 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b82a3ca-360c-4bc9-9231-f07821d4bf89 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:15.768649 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b82a3ca-360c-4bc9-9231-f07821d4bf89 None admin] PATH_INFO: `/v3/services` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:15.769566 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b82a3ca-360c-4bc9-9231-f07821d4bf89 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:15.770091 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b82a3ca-360c-4bc9-9231-f07821d4bf89 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:15.771658 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b82a3ca-360c-4bc9-9231-f07821d4bf89 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:15.778290 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 468/929] 10.222.0.58 () {56 vars in 1126 bytes} [Tue Jul 26 13:34:15 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:15.784689 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0038fbd-d334-4c5e-bac6-e6f4293ca354 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:15.798994 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:15.902794 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0038fbd-d334-4c5e-bac6-e6f4293ca354 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:15.909270 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8e7bc7-3235-4887-b9dc-da875696a811 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:15.909779 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8e7bc7-3235-4887-b9dc-da875696a811 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:15.909887 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8e7bc7-3235-4887-b9dc-da875696a811 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:15.911439 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 469/930] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:34:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:34:15.917013 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408489ac-5e5e-4b95-9014-acb642caa501 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:15.917296 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408489ac-5e5e-4b95-9014-acb642caa501 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:15.917604 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408489ac-5e5e-4b95-9014-acb642caa501 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:15.938076 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-408489ac-5e5e-4b95-9014-acb642caa501 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:34:15.939285 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-408489ac-5e5e-4b95-9014-acb642caa501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:34:15.940462 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f0e967-ffa1-4568-b02e-09e103ff932c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:15.941095 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f0e967-ffa1-4568-b02e-09e103ff932c None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:15.941511 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f0e967-ffa1-4568-b02e-09e103ff932c None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:15.941796 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f0e967-ffa1-4568-b02e-09e103ff932c None admin] PATH_INFO: `/v3/services` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:15.942608 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f0e967-ffa1-4568-b02e-09e103ff932c None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:15.942912 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f0e967-ffa1-4568-b02e-09e103ff932c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:15.944294 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f0e967-ffa1-4568-b02e-09e103ff932c None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:15.950174 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 462/931] 10.222.0.58 () {56 vars in 1126 bytes} [Tue Jul 26 13:34:15 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:15.956550 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f0e967-ffa1-4568-b02e-09e103ff932c None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:16.040535 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:16.041911 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 470/932] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:34:15 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 13:34:16.063763 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408489ac-5e5e-4b95-9014-acb642caa501 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:16.074389 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f0e967-ffa1-4568-b02e-09e103ff932c None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:16.111367 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69db1beb-73b7-4f7c-82dc-ffa7fddc7747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:16.111992 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69db1beb-73b7-4f7c-82dc-ffa7fddc7747 None admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:16.112266 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69db1beb-73b7-4f7c-82dc-ffa7fddc7747 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:16.112540 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69db1beb-73b7-4f7c-82dc-ffa7fddc7747 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:16.113247 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69db1beb-73b7-4f7c-82dc-ffa7fddc7747 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:16.114657 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69db1beb-73b7-4f7c-82dc-ffa7fddc7747 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:16.139446 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 463/933] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 13:34:15 2022] POST /identity/v3/endpoints => generated 347 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:34:16.141836 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:16.197611 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408489ac-5e5e-4b95-9014-acb642caa501 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:16.263793 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3cb999-cbc4-4bec-994d-708f062f6396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:16.265218 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3cb999-cbc4-4bec-994d-708f062f6396 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:16.265979 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3cb999-cbc4-4bec-994d-708f062f6396 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:16.266697 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3cb999-cbc4-4bec-994d-708f062f6396 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:16.289156 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3cb999-cbc4-4bec-994d-708f062f6396 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:34:16.300166 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-8d3cb999-cbc4-4bec-994d-708f062f6396 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 13:34:16.300166 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:16.300166 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:34:16.300166 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:34:16.300166 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:34:16.300166 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:34:16.300166 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:34:16.300166 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:34:16.300166 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:34:16.300166 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:34:16.300166 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:34:16.300166 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:34:16.300166 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:34:16.300166 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:34:16.300166 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:34:16.300166 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:34:16.302393 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:34:16.302393 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:34:16.302393 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:34:16.302393 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:34:16.302393 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:34:16.302393 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:34:16.302393 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:34:16.302393 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:34:16.302393 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:34:16.302393 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:34:16.302393 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:34:16.302393 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:34:16.302393 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:34:16.302393 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:34:16.302393 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:34:16.302393 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:34:16.304488 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:34:16.304488 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:34:16.304488 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:34:16.304488 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:34:16.304488 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:34:16.304488 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:34:16.304488 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:34:16.304488 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:34:16.304488 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:34:16.304488 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:34:16.304488 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 13:34:16.304488 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:34:16.308873 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 471/934] 10.222.0.58 () {56 vars in 1120 bytes} [Tue Jul 26 13:34:16 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:34:16.318225 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69db1beb-73b7-4f7c-82dc-ffa7fddc7747 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:16.437081 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69db1beb-73b7-4f7c-82dc-ffa7fddc7747 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:16.469706 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:16.482003 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35276a41-de26-4e3d-8134-c3b3a4c429a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:16.483091 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35276a41-de26-4e3d-8134-c3b3a4c429a8 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:16.483555 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35276a41-de26-4e3d-8134-c3b3a4c429a8 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:16.483993 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35276a41-de26-4e3d-8134-c3b3a4c429a8 None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:16.484965 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35276a41-de26-4e3d-8134-c3b3a4c429a8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:16.485502 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35276a41-de26-4e3d-8134-c3b3a4c429a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:16.487398 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35276a41-de26-4e3d-8134-c3b3a4c429a8 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:16.497449 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 464/935] 10.222.0.58 () {56 vars in 1115 bytes} [Tue Jul 26 13:34:16 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:16.503357 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3cb999-cbc4-4bec-994d-708f062f6396 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:16.611025 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3cb999-cbc4-4bec-994d-708f062f6396 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:16.630104 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:16.649331 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1443f44d-cdf6-429f-bf22-36ba879f7a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:16.650295 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1443f44d-cdf6-429f-bf22-36ba879f7a29 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:16.650730 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1443f44d-cdf6-429f-bf22-36ba879f7a29 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:16.651156 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1443f44d-cdf6-429f-bf22-36ba879f7a29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:16.662448 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1443f44d-cdf6-429f-bf22-36ba879f7a29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:16.663971 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1443f44d-cdf6-429f-bf22-36ba879f7a29 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:16.689187 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-1443f44d-cdf6-429f-bf22-36ba879f7a29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:34:16.689187 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:16.689187 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:34:16.689187 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:34:16.689187 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:34:16.689187 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:34:16.689187 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:34:16.689187 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:34:16.689187 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:34:16.689187 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:34:16.689187 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:34:16.689187 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:34:16.689187 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 13:34:16.689187 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 13:34:16.689187 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 13:34:16.690267 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 13:34:16.690267 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:34:16.690267 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:34:16.690267 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:34:16.690267 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:34:16.690267 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:34:16.690267 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:34:16.690267 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:34:16.690267 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:34:16.690267 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:34:16.690267 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:34:16.690267 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:34:16.690267 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:34:16.690267 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:34:16.690267 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:34:16.690267 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:34:16.690267 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:34:16.691188 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 13:34:16.691188 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 13:34:16.691188 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:34:16.691188 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:34:16.691188 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 472/936] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:34:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:34:16.698666 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35276a41-de26-4e3d-8134-c3b3a4c429a8 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:16.814497 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35276a41-de26-4e3d-8134-c3b3a4c429a8 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:16.832644 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add3f7e3-24c0-4084-9283-de46de190dbe None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:16.833030 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add3f7e3-24c0-4084-9283-de46de190dbe None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:16.833527 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add3f7e3-24c0-4084-9283-de46de190dbe None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:16.835430 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 473/937] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:34:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 13:34:16.840545 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4758c9bf-2414-4bb9-90a6-dba50799073b None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:16.840831 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4758c9bf-2414-4bb9-90a6-dba50799073b None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:16.841108 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4758c9bf-2414-4bb9-90a6-dba50799073b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:16.853917 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1a1da2-d173-4a46-a666-1ece559eb114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:16.854835 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1a1da2-d173-4a46-a666-1ece559eb114 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:16.855271 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1a1da2-d173-4a46-a666-1ece559eb114 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:16.855692 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1a1da2-d173-4a46-a666-1ece559eb114 None admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:16.856543 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1a1da2-d173-4a46-a666-1ece559eb114 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:16.857198 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1a1da2-d173-4a46-a666-1ece559eb114 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:16.858789 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1a1da2-d173-4a46-a666-1ece559eb114 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:16.862119 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-4758c9bf-2414-4bb9-90a6-dba50799073b None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:34:16.862980 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-4758c9bf-2414-4bb9-90a6-dba50799073b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:34:16.877586 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 465/938] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Jul 26 13:34:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:16.879321 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1a1da2-d173-4a46-a666-1ece559eb114 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:16.938224 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:16.948702 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 474/939] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:34:16 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 13:34:16.971648 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4758c9bf-2414-4bb9-90a6-dba50799073b None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:17.003676 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1a1da2-d173-4a46-a666-1ece559eb114 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:17.046853 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c005eb10-8cac-4f5b-b452-b282319de3c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:17.047874 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c005eb10-8cac-4f5b-b452-b282319de3c9 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:17.048365 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c005eb10-8cac-4f5b-b452-b282319de3c9 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:17.048978 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c005eb10-8cac-4f5b-b452-b282319de3c9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:17.058155 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c005eb10-8cac-4f5b-b452-b282319de3c9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:17.060009 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c005eb10-8cac-4f5b-b452-b282319de3c9 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:17.073502 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-c005eb10-8cac-4f5b-b452-b282319de3c9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 13:34:17.073502 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:17.073502 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:34:17.073502 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:34:17.073502 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:34:17.073502 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:34:17.073502 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:34:17.073502 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:34:17.073502 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:34:17.073502 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:34:17.073502 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:34:17.073502 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:34:17.073502 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 13:34:17.073502 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 13:34:17.073502 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 13:34:17.073502 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 13:34:17.074466 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:34:17.074466 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:34:17.074466 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 13:34:17.074466 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 13:34:17.074466 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 13:34:17.074466 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 13:34:17.074466 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:34:17.074466 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:34:17.074466 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:34:17.074466 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:34:17.074466 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:34:17.074466 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:34:17.074466 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:34:17.074466 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:34:17.074466 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:34:17.074466 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:34:17.074466 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:34:17.074466 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:34:17.075422 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:34:17.075422 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:34:17.075422 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 13:34:17.075422 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 13:34:17.075422 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 13:34:17.075422 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 13:34:17.075422 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 13:34:17.075422 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 13:34:17.075422 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 13:34:17.075422 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:34:17.078134 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 466/940] 10.222.0.58 () {56 vars in 1117 bytes} [Tue Jul 26 13:34:16 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:34:17.084311 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c005eb10-8cac-4f5b-b452-b282319de3c9 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:17.100192 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:17.104608 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4758c9bf-2414-4bb9-90a6-dba50799073b None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:17.152011 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4930e71-850a-497a-aba5-1f55cd08184d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:17.152965 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4930e71-850a-497a-aba5-1f55cd08184d None admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:17.153418 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4930e71-850a-497a-aba5-1f55cd08184d None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:17.153882 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4930e71-850a-497a-aba5-1f55cd08184d None admin] PATH_INFO: `/v3/projects` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:17.154802 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4930e71-850a-497a-aba5-1f55cd08184d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:17.156398 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4930e71-850a-497a-aba5-1f55cd08184d None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:17.159192 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: WARNING keystone.server.flask.common [None req-f4930e71-850a-497a-aba5-1f55cd08184d None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 26 13:34:17.177335 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.sql.core [None req-f4930e71-850a-497a-aba5-1f55cd08184d None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 26 13:34:17.177335 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 26 13:34:17.177335 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [parameters: {'id': '7fe3ea065f1845448c31ddf35686ea44', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 26 13:34:17.177335 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=86933) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 26 13:34:17.186255 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-f4930e71-850a-497a-aba5-1f55cd08184d None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7fe3ea065f1845448c31ddf35686ea44.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7fe3ea065f1845448c31ddf35686ea44. Jul 26 13:34:17.186255 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:17.186255 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 26 13:34:17.186255 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 26 13:34:17.186255 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 26 13:34:17.186255 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 26 13:34:17.186255 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 26 13:34:17.186255 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application result = self._query(query) Jul 26 13:34:17.186255 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 26 13:34:17.186255 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application conn.query(q) Jul 26 13:34:17.186255 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 26 13:34:17.186255 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 26 13:34:17.186255 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 26 13:34:17.187073 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application result.read() Jul 26 13:34:17.187073 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 26 13:34:17.187073 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 26 13:34:17.187073 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 26 13:34:17.187073 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 26 13:34:17.187073 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 26 13:34:17.187073 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 26 13:34:17.187073 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 26 13:34:17.187073 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 26 13:34:17.187073 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 26 13:34:17.187073 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:34:17.187073 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 26 13:34:17.187073 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:34:17.187073 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:17.187073 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 26 13:34:17.187073 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 26 13:34:17.187073 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 26 13:34:17.187073 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 26 13:34:17.188019 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 26 13:34:17.188019 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application next(self.gen) Jul 26 13:34:17.188019 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 26 13:34:17.188019 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application yield resource Jul 26 13:34:17.188019 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 26 13:34:17.188019 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application next(self.gen) Jul 26 13:34:17.188019 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 26 13:34:17.188019 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application self.session.rollback() Jul 26 13:34:17.188019 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 26 13:34:17.188019 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application self.force_reraise() Jul 26 13:34:17.188019 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 26 13:34:17.188019 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise self.value Jul 26 13:34:17.188019 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 26 13:34:17.188019 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 26 13:34:17.188019 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 26 13:34:17.188019 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application session.commit() Jul 26 13:34:17.188019 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 26 13:34:17.188902 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 26 13:34:17.188902 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 26 13:34:17.188902 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application self._prepare_impl() Jul 26 13:34:17.188902 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 26 13:34:17.188902 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application self.session.flush() Jul 26 13:34:17.188902 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 26 13:34:17.188902 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application self._flush(objects) Jul 26 13:34:17.188902 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 26 13:34:17.188902 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 26 13:34:17.188902 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 26 13:34:17.188902 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application compat.raise_( Jul 26 13:34:17.188902 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 26 13:34:17.188902 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception Jul 26 13:34:17.188902 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 26 13:34:17.188902 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application flush_context.execute() Jul 26 13:34:17.188902 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 26 13:34:17.188902 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rec.execute(self) Jul 26 13:34:17.189824 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 26 13:34:17.189824 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 26 13:34:17.189824 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 26 13:34:17.189824 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 26 13:34:17.189824 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 26 13:34:17.189824 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 26 13:34:17.189824 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 26 13:34:17.189824 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 26 13:34:17.189824 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 26 13:34:17.189824 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 26 13:34:17.189824 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 26 13:34:17.189824 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 26 13:34:17.189824 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 26 13:34:17.189824 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 26 13:34:17.189824 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 26 13:34:17.190617 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 26 13:34:17.190617 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 26 13:34:17.190617 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception Jul 26 13:34:17.190617 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 26 13:34:17.190617 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 26 13:34:17.190617 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 26 13:34:17.190617 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 26 13:34:17.190617 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 26 13:34:17.190617 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application result = self._query(query) Jul 26 13:34:17.190617 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 26 13:34:17.190617 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application conn.query(q) Jul 26 13:34:17.190617 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 26 13:34:17.190617 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 26 13:34:17.190617 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 26 13:34:17.190617 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application result.read() Jul 26 13:34:17.190617 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 26 13:34:17.190617 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 26 13:34:17.191460 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 26 13:34:17.191460 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 26 13:34:17.191460 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 26 13:34:17.191460 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 26 13:34:17.191460 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 26 13:34:17.191460 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 26 13:34:17.191460 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 26 13:34:17.191460 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 26 13:34:17.191460 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [parameters: {'id': '7fe3ea065f1845448c31ddf35686ea44', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 26 13:34:17.191460 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 26 13:34:17.191460 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:34:17.191460 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 13:34:17.191460 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:34:17.191460 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:17.191460 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 26 13:34:17.192242 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 26 13:34:17.192242 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 26 13:34:17.192242 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 26 13:34:17.192242 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 26 13:34:17.192242 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:34:17.192242 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 13:34:17.192242 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:34:17.192242 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:17.192242 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:34:17.192242 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:34:17.192242 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:34:17.192242 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:34:17.192242 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:34:17.192242 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:34:17.192242 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:34:17.192242 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:34:17.192242 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:34:17.193147 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:34:17.193147 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 26 13:34:17.193147 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 26 13:34:17.193147 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:34:17.193147 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:34:17.193147 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 26 13:34:17.193147 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 26 13:34:17.193147 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7fe3ea065f1845448c31ddf35686ea44. Jul 26 13:34:17.193147 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:34:17.195750 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 475/941] 10.222.0.58 () {60 vars in 1140 bytes} [Tue Jul 26 13:34:16 2022] POST /identity/v3/projects => generated 262 bytes in 224 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 26 13:34:17.201921 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4930e71-850a-497a-aba5-1f55cd08184d None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:17.218519 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c005eb10-8cac-4f5b-b452-b282319de3c9 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:17.229218 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:17.257311 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b909dabb-9963-4b03-ac39-6c82b24b1f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:17.258094 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b909dabb-9963-4b03-ac39-6c82b24b1f89 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:17.258480 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b909dabb-9963-4b03-ac39-6c82b24b1f89 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:17.258918 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b909dabb-9963-4b03-ac39-6c82b24b1f89 None admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:17.259889 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b909dabb-9963-4b03-ac39-6c82b24b1f89 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:17.260301 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b909dabb-9963-4b03-ac39-6c82b24b1f89 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:17.262047 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b909dabb-9963-4b03-ac39-6c82b24b1f89 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:17.287052 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 467/942] 10.222.0.58 () {56 vars in 1149 bytes} [Tue Jul 26 13:34:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:17.289056 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b909dabb-9963-4b03-ac39-6c82b24b1f89 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:17.319617 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4930e71-850a-497a-aba5-1f55cd08184d None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:17.358242 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6a0c93-9fea-4006-a00d-409e9dbdb9b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:17.359333 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6a0c93-9fea-4006-a00d-409e9dbdb9b6 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:17.359831 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6a0c93-9fea-4006-a00d-409e9dbdb9b6 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:17.360392 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6a0c93-9fea-4006-a00d-409e9dbdb9b6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:17.369712 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6a0c93-9fea-4006-a00d-409e9dbdb9b6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:17.371424 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6a0c93-9fea-4006-a00d-409e9dbdb9b6 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:17.384794 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-5d6a0c93-9fea-4006-a00d-409e9dbdb9b6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 13:34:17.384794 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:17.384794 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:34:17.384794 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:34:17.384794 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:34:17.384794 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:34:17.384794 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:34:17.384794 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:34:17.384794 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:34:17.384794 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:34:17.384794 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:34:17.384794 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:34:17.384794 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 13:34:17.384794 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 13:34:17.384794 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 13:34:17.385980 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 13:34:17.385980 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:34:17.385980 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:34:17.385980 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:34:17.385980 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:34:17.385980 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:34:17.385980 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:34:17.385980 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:34:17.385980 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:34:17.385980 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:34:17.385980 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:34:17.385980 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:34:17.385980 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:34:17.385980 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:34:17.385980 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:34:17.385980 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:34:17.385980 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:34:17.386951 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 13:34:17.386951 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 13:34:17.386951 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 13:34:17.386951 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 13:34:17.386951 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 13:34:17.386951 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 13:34:17.386951 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 13:34:17.386951 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:34:17.386951 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 476/943] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Jul 26 13:34:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:34:17.387864 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5353661e-6659-46b5-a2ca-d758bda2cce1 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:17.388150 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5353661e-6659-46b5-a2ca-d758bda2cce1 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:17.388428 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5353661e-6659-46b5-a2ca-d758bda2cce1 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:17.392910 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 477/944] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:34:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:34:17.393687 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5353661e-6659-46b5-a2ca-d758bda2cce1 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:17.414384 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:17.417540 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b909dabb-9963-4b03-ac39-6c82b24b1f89 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:17.465431 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039b7007-ab2e-47eb-847e-865bec5e49b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:17.466162 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039b7007-ab2e-47eb-847e-865bec5e49b0 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:17.466748 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039b7007-ab2e-47eb-847e-865bec5e49b0 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:17.467164 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039b7007-ab2e-47eb-847e-865bec5e49b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:17.479828 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039b7007-ab2e-47eb-847e-865bec5e49b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:17.481595 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039b7007-ab2e-47eb-847e-865bec5e49b0 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:17.504089 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-039b7007-ab2e-47eb-847e-865bec5e49b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:34:17.504089 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:17.504089 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:34:17.504089 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:34:17.504089 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:34:17.504089 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:34:17.504089 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:34:17.504089 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:34:17.504089 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:34:17.504089 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:34:17.504089 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:34:17.504089 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:34:17.504089 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 13:34:17.504089 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 13:34:17.504089 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 13:34:17.513415 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 13:34:17.513415 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:34:17.513415 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:34:17.513415 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:34:17.513415 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:34:17.513415 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:34:17.513415 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:34:17.513415 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:34:17.513415 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:34:17.513415 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:34:17.513415 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:34:17.513415 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:34:17.513415 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:34:17.513415 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:34:17.513415 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:34:17.513415 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:34:17.513415 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:34:17.515059 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 13:34:17.515059 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 13:34:17.515059 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:34:17.515059 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:34:17.515059 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 468/945] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:34:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:34:17.515059 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67460e2a-aac1-49f5-87b3-e1b6c9449f97 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:17.515059 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67460e2a-aac1-49f5-87b3-e1b6c9449f97 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:17.515059 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67460e2a-aac1-49f5-87b3-e1b6c9449f97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:17.532297 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-67460e2a-aac1-49f5-87b3-e1b6c9449f97 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:34:17.533510 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-67460e2a-aac1-49f5-87b3-e1b6c9449f97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:34:17.535923 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5353661e-6659-46b5-a2ca-d758bda2cce1 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:17.548909 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:17.573521 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df821128-49fd-419f-8a1d-44b3ce833eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:17.574838 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df821128-49fd-419f-8a1d-44b3ce833eba None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:17.574838 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df821128-49fd-419f-8a1d-44b3ce833eba None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:17.574838 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df821128-49fd-419f-8a1d-44b3ce833eba None admin] PATH_INFO: `/v3/projects` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:17.575589 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df821128-49fd-419f-8a1d-44b3ce833eba None admin] RBAC: Adding query filter params (name=service) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:17.575903 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df821128-49fd-419f-8a1d-44b3ce833eba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:17.577318 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df821128-49fd-419f-8a1d-44b3ce833eba None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:17.597554 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 478/946] 10.222.0.58 () {56 vars in 1112 bytes} [Tue Jul 26 13:34:17 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:17.598416 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df821128-49fd-419f-8a1d-44b3ce833eba None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:17.622286 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 469/947] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:34:17 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 13:34:17.646791 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67460e2a-aac1-49f5-87b3-e1b6c9449f97 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:17.705358 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:17.735531 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df821128-49fd-419f-8a1d-44b3ce833eba None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:17.770526 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67460e2a-aac1-49f5-87b3-e1b6c9449f97 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:17.781039 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60dd340-e725-4197-8620-676c20b45f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:17.781719 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60dd340-e725-4197-8620-676c20b45f8a None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:17.782101 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60dd340-e725-4197-8620-676c20b45f8a None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:17.782442 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60dd340-e725-4197-8620-676c20b45f8a None admin] PATH_INFO: `/v3/domains` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:17.783172 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60dd340-e725-4197-8620-676c20b45f8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:17.783489 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60dd340-e725-4197-8620-676c20b45f8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:17.784978 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60dd340-e725-4197-8620-676c20b45f8a None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:17.798681 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 479/948] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Jul 26 13:34:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:17.804860 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60dd340-e725-4197-8620-676c20b45f8a None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:17.811995 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f941d75-33ad-4c89-b1e8-505c2245a9eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:17.812821 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f941d75-33ad-4c89-b1e8-505c2245a9eb None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:17.813208 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f941d75-33ad-4c89-b1e8-505c2245a9eb None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:17.813617 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f941d75-33ad-4c89-b1e8-505c2245a9eb None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:17.814370 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f941d75-33ad-4c89-b1e8-505c2245a9eb None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:17.816087 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f941d75-33ad-4c89-b1e8-505c2245a9eb None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:17.828320 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-1f941d75-33ad-4c89-b1e8-505c2245a9eb None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 13:34:17.828320 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:17.828320 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 13:34:17.828320 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 13:34:17.828320 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 13:34:17.828320 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 13:34:17.828320 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 13:34:17.828320 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 13:34:17.828320 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 13:34:17.828320 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:34:17.828320 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 13:34:17.828320 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:34:17.828320 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:17.828320 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:34:17.828320 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:34:17.828320 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:34:17.828320 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:34:17.829600 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:34:17.829600 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:34:17.829600 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:34:17.829600 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:34:17.829600 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:34:17.829600 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:34:17.829600 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 13:34:17.829600 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 13:34:17.829600 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 13:34:17.829600 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 13:34:17.829600 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:34:17.829600 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:34:17.829600 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:34:17.829600 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:34:17.829600 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:34:17.829600 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:34:17.830450 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:34:17.830450 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:34:17.830450 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:34:17.830450 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:34:17.830450 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:34:17.830450 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:34:17.830450 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:34:17.830450 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:34:17.830450 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:34:17.830450 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:34:17.830450 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 13:34:17.830450 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 13:34:17.830450 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 13:34:17.830450 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:34:17.830450 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 470/949] 10.222.0.58 () {56 vars in 1126 bytes} [Tue Jul 26 13:34:17 2022] GET /identity/v3/services/object-store => generated 93 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 13:34:17.836561 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f941d75-33ad-4c89-b1e8-505c2245a9eb None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:17.855548 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:17.933560 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60dd340-e725-4197-8620-676c20b45f8a None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:17.965777 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f941d75-33ad-4c89-b1e8-505c2245a9eb None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:17.971809 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a959a33f-fe7d-417e-a926-5acb4a5b58ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:17.972598 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a959a33f-fe7d-417e-a926-5acb4a5b58ea None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:17.972976 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a959a33f-fe7d-417e-a926-5acb4a5b58ea None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:17.973351 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a959a33f-fe7d-417e-a926-5acb4a5b58ea None admin] PATH_INFO: `/v3/projects/service` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:17.976990 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:17.984158 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a959a33f-fe7d-417e-a926-5acb4a5b58ea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:17.987684 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a959a33f-fe7d-417e-a926-5acb4a5b58ea None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:18.005875 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-a959a33f-fe7d-417e-a926-5acb4a5b58ea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 13:34:18.005875 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:18.005875 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:34:18.005875 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:34:18.005875 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:34:18.005875 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:34:18.005875 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:34:18.005875 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:34:18.005875 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:34:18.005875 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:34:18.005875 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:34:18.005875 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:34:18.005875 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 13:34:18.005875 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 13:34:18.005875 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 13:34:18.007571 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 13:34:18.007571 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:34:18.007571 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:34:18.007571 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:34:18.007571 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:34:18.007571 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:34:18.007571 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:34:18.007571 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:34:18.007571 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:34:18.007571 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:34:18.007571 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:34:18.007571 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:34:18.007571 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:34:18.007571 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:34:18.007571 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:34:18.007571 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:34:18.007571 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:34:18.008643 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 13:34:18.008643 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 13:34:18.008643 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 13:34:18.008643 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 13:34:18.008643 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 13:34:18.008643 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 13:34:18.008643 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 13:34:18.008643 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:34:18.010817 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 480/950] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Jul 26 13:34:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:34:18.012255 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e633ad4e-3b87-42c8-a666-a81d03fe3342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:18.012991 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e633ad4e-3b87-42c8-a666-a81d03fe3342 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:18.013369 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e633ad4e-3b87-42c8-a666-a81d03fe3342 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:18.017706 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e633ad4e-3b87-42c8-a666-a81d03fe3342 None admin] PATH_INFO: `/v3/services` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:18.017706 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e633ad4e-3b87-42c8-a666-a81d03fe3342 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:18.017706 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e633ad4e-3b87-42c8-a666-a81d03fe3342 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:18.018017 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a959a33f-fe7d-417e-a926-5acb4a5b58ea None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:18.018101 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e633ad4e-3b87-42c8-a666-a81d03fe3342 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:18.022977 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 471/951] 10.222.0.58 () {56 vars in 1122 bytes} [Tue Jul 26 13:34:17 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:18.028760 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e633ad4e-3b87-42c8-a666-a81d03fe3342 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:18.127336 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:18.143248 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a959a33f-fe7d-417e-a926-5acb4a5b58ea None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:18.159399 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e633ad4e-3b87-42c8-a666-a81d03fe3342 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:18.180501 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f46172a-118f-4d03-9258-e8523f3297e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:18.181716 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f46172a-118f-4d03-9258-e8523f3297e3 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:18.182253 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f46172a-118f-4d03-9258-e8523f3297e3 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:18.182729 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f46172a-118f-4d03-9258-e8523f3297e3 None admin] PATH_INFO: `/v3/projects` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:18.183746 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f46172a-118f-4d03-9258-e8523f3297e3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:18.184289 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f46172a-118f-4d03-9258-e8523f3297e3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:18.186091 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f46172a-118f-4d03-9258-e8523f3297e3 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:18.200971 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 481/952] 10.222.0.58 () {56 vars in 1148 bytes} [Tue Jul 26 13:34:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:18.202803 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95462f9-9711-4ffa-b24a-368fa24606fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:18.203664 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95462f9-9711-4ffa-b24a-368fa24606fa None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:18.204148 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95462f9-9711-4ffa-b24a-368fa24606fa None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:18.204652 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95462f9-9711-4ffa-b24a-368fa24606fa None admin] PATH_INFO: `/v3/services` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:18.205579 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95462f9-9711-4ffa-b24a-368fa24606fa None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:18.205992 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95462f9-9711-4ffa-b24a-368fa24606fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:18.207488 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95462f9-9711-4ffa-b24a-368fa24606fa None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:18.211177 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f46172a-118f-4d03-9258-e8523f3297e3 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:18.218519 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 472/953] 10.222.0.58 () {56 vars in 1122 bytes} [Tue Jul 26 13:34:18 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:34:18.219860 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95462f9-9711-4ffa-b24a-368fa24606fa None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:18.298740 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:18.332537 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95462f9-9711-4ffa-b24a-368fa24606fa None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:18.355718 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f46172a-118f-4d03-9258-e8523f3297e3 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:18.370097 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a11a205-adca-48cb-8f9e-af838da3a62e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:18.370813 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a11a205-adca-48cb-8f9e-af838da3a62e None admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:18.371103 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a11a205-adca-48cb-8f9e-af838da3a62e None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:18.371625 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a11a205-adca-48cb-8f9e-af838da3a62e None admin] PATH_INFO: `/v3/endpoints` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:18.372083 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a11a205-adca-48cb-8f9e-af838da3a62e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:18.373451 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a11a205-adca-48cb-8f9e-af838da3a62e None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:18.383624 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:18.393268 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bae1db-1cf2-4c94-9bf1-ce697c2a134f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:18.394310 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bae1db-1cf2-4c94-9bf1-ce697c2a134f None admin] REQUEST_METHOD: `PUT` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:18.394792 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bae1db-1cf2-4c94-9bf1-ce697c2a134f None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:18.395284 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bae1db-1cf2-4c94-9bf1-ce697c2a134f None admin] PATH_INFO: `/v3/projects/1e8bb4385c204210a1bc77b1f0d654f3/users/b64bd7b183d24d8c9037b261c7884810/roles/ac11ff5f40194d7d98f81a5429aa3826` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:18.397979 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bae1db-1cf2-4c94-9bf1-ce697c2a134f None admin] RBAC: Authorizing `identity:create_grant(project_id=1e8bb4385c204210a1bc77b1f0d654f3, user_id=b64bd7b183d24d8c9037b261c7884810, role_id=ac11ff5f40194d7d98f81a5429aa3826)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:18.399725 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bae1db-1cf2-4c94-9bf1-ce697c2a134f None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:18.405679 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 473/954] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 13:34:18 2022] POST /identity/v3/endpoints => generated 349 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 13:34:18.407231 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e2c9c8-f8af-4bd6-90ac-8266848cc60e None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:18.407522 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e2c9c8-f8af-4bd6-90ac-8266848cc60e None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:18.407792 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e2c9c8-f8af-4bd6-90ac-8266848cc60e None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:18.409272 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 474/955] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:34:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:34:18.425579 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbe69b5-cc43-4cda-89a3-1b0fefbce7ab None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:18.425579 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbe69b5-cc43-4cda-89a3-1b0fefbce7ab None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:18.425579 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbe69b5-cc43-4cda-89a3-1b0fefbce7ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:18.426504 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 482/956] 10.222.0.58 () {58 vars in 1439 bytes} [Tue Jul 26 13:34:18 2022] PUT /identity/v3/projects/1e8bb4385c204210a1bc77b1f0d654f3/users/b64bd7b183d24d8c9037b261c7884810/roles/ac11ff5f40194d7d98f81a5429aa3826 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 13:34:18.444786 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-4dbe69b5-cc43-4cda-89a3-1b0fefbce7ab None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:34:18.446002 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-4dbe69b5-cc43-4cda-89a3-1b0fefbce7ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:34:18.550041 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 475/957] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:34:18 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 13:34:18.582511 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bae1db-1cf2-4c94-9bf1-ce697c2a134f None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:18.731193 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bae1db-1cf2-4c94-9bf1-ce697c2a134f None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:18.750550 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:18.775038 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079561de-f43d-4d36-b9c2-dd940d6bba66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:18.775709 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079561de-f43d-4d36-b9c2-dd940d6bba66 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:18.775996 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079561de-f43d-4d36-b9c2-dd940d6bba66 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:18.776277 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079561de-f43d-4d36-b9c2-dd940d6bba66 None admin] PATH_INFO: `/v3/services/compute` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:18.776917 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079561de-f43d-4d36-b9c2-dd940d6bba66 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:18.778340 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079561de-f43d-4d36-b9c2-dd940d6bba66 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:18.788719 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-079561de-f43d-4d36-b9c2-dd940d6bba66 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 26 13:34:18.788719 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:18.788719 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 13:34:18.788719 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 13:34:18.788719 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 13:34:18.788719 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 13:34:18.788719 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 13:34:18.788719 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 13:34:18.788719 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 26 13:34:18.788719 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:34:18.788719 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 13:34:18.788719 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:34:18.788719 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:18.788719 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:34:18.788719 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:34:18.788719 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:34:18.788719 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:34:18.790884 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:34:18.790884 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:34:18.790884 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:34:18.790884 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:34:18.790884 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:34:18.790884 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:34:18.790884 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 13:34:18.790884 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 13:34:18.790884 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 13:34:18.790884 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 13:34:18.790884 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:34:18.790884 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:34:18.790884 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:34:18.790884 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:34:18.790884 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:34:18.790884 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:34:18.792598 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:34:18.792598 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:34:18.792598 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:34:18.792598 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:34:18.792598 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:34:18.792598 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:34:18.792598 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:34:18.792598 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:34:18.792598 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:34:18.792598 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:34:18.792598 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 13:34:18.792598 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 13:34:18.792598 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 26 13:34:18.792598 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:34:18.792598 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 483/958] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Jul 26 13:34:18 2022] GET /identity/v3/services/compute => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 13:34:18.796799 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbe69b5-cc43-4cda-89a3-1b0fefbce7ab None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:18.916814 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbe69b5-cc43-4cda-89a3-1b0fefbce7ab None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:18.943252 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:18.954902 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750a49fa-33ee-46b5-bd75-1bfaece85d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:18.955531 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750a49fa-33ee-46b5-bd75-1bfaece85d5c None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:18.955810 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750a49fa-33ee-46b5-bd75-1bfaece85d5c None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:18.956083 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750a49fa-33ee-46b5-bd75-1bfaece85d5c None admin] PATH_INFO: `/v3/services` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:18.956767 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750a49fa-33ee-46b5-bd75-1bfaece85d5c None admin] RBAC: Adding query filter params (name=compute) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:18.957066 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750a49fa-33ee-46b5-bd75-1bfaece85d5c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:18.958566 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750a49fa-33ee-46b5-bd75-1bfaece85d5c None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:18.964768 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 476/959] 10.222.0.58 () {56 vars in 1112 bytes} [Tue Jul 26 13:34:18 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:18.970589 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079561de-f43d-4d36-b9c2-dd940d6bba66 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:19.091546 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079561de-f43d-4d36-b9c2-dd940d6bba66 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:19.126548 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79828fa-fd73-4bb5-b1be-a5f51abbb86f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:19.127237 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79828fa-fd73-4bb5-b1be-a5f51abbb86f None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:19.127516 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79828fa-fd73-4bb5-b1be-a5f51abbb86f None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:19.127790 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79828fa-fd73-4bb5-b1be-a5f51abbb86f None admin] PATH_INFO: `/v3/services` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:19.128472 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79828fa-fd73-4bb5-b1be-a5f51abbb86f None admin] RBAC: Adding query filter params (type=compute) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:19.128777 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79828fa-fd73-4bb5-b1be-a5f51abbb86f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:19.130166 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79828fa-fd73-4bb5-b1be-a5f51abbb86f None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:19.135968 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 484/960] 10.222.0.58 () {56 vars in 1112 bytes} [Tue Jul 26 13:34:18 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:19.857403 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3154802-f5d3-42c2-a68f-f4554a877989 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:19.857791 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3154802-f5d3-42c2-a68f-f4554a877989 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:19.858108 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3154802-f5d3-42c2-a68f-f4554a877989 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:19.859768 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 477/961] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:34:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:34:19.865036 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812bb4a3-88bd-47c4-98c5-9887ac30c313 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:19.865338 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812bb4a3-88bd-47c4-98c5-9887ac30c313 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:19.865651 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812bb4a3-88bd-47c4-98c5-9887ac30c313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:19.885018 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-812bb4a3-88bd-47c4-98c5-9887ac30c313 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:34:19.886591 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-812bb4a3-88bd-47c4-98c5-9887ac30c313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:34:19.928389 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:19.961640 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 485/962] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:34:19 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 13:34:19.981703 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3154802-f5d3-42c2-a68f-f4554a877989 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:20.101874 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3154802-f5d3-42c2-a68f-f4554a877989 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:20.138264 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685b9cc0-0d7d-414a-9d06-5ae8cc0db45e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:20.138764 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685b9cc0-0d7d-414a-9d06-5ae8cc0db45e None admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:20.139047 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685b9cc0-0d7d-414a-9d06-5ae8cc0db45e None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:20.139325 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685b9cc0-0d7d-414a-9d06-5ae8cc0db45e None admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:20.140016 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685b9cc0-0d7d-414a-9d06-5ae8cc0db45e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:20.141450 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685b9cc0-0d7d-414a-9d06-5ae8cc0db45e None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:20.144049 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: WARNING keystone.server.flask.common [None req-685b9cc0-0d7d-414a-9d06-5ae8cc0db45e None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 26 13:34:20.157991 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.sql.core [None req-685b9cc0-0d7d-414a-9d06-5ae8cc0db45e None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 26 13:34:20.157991 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 26 13:34:20.157991 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [parameters: {'user_id': '81a5b870472249329622e6da96c814f1', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 26 13:34:20.157991 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=86934) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 26 13:34:20.176316 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-685b9cc0-0d7d-414a-9d06-5ae8cc0db45e None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 26 13:34:20.176316 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:20.176316 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 26 13:34:20.176316 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 26 13:34:20.176316 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 26 13:34:20.176316 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 26 13:34:20.176316 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 26 13:34:20.176316 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application result = self._query(query) Jul 26 13:34:20.176316 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 26 13:34:20.176316 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application conn.query(q) Jul 26 13:34:20.176316 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 26 13:34:20.176316 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 26 13:34:20.176316 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 26 13:34:20.176316 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application result.read() Jul 26 13:34:20.176316 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 26 13:34:20.177423 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 26 13:34:20.177423 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 26 13:34:20.177423 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 26 13:34:20.177423 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 26 13:34:20.177423 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 26 13:34:20.177423 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 26 13:34:20.177423 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 26 13:34:20.177423 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 26 13:34:20.177423 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:34:20.177423 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 26 13:34:20.177423 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:34:20.177423 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:20.177423 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 26 13:34:20.177423 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 26 13:34:20.177423 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 26 13:34:20.177423 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 26 13:34:20.177423 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 26 13:34:20.177423 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application next(self.gen) Jul 26 13:34:20.178441 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 26 13:34:20.178441 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application yield resource Jul 26 13:34:20.178441 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 26 13:34:20.178441 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application next(self.gen) Jul 26 13:34:20.178441 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 26 13:34:20.178441 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application self.session.rollback() Jul 26 13:34:20.178441 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 26 13:34:20.178441 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application self.force_reraise() Jul 26 13:34:20.178441 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 26 13:34:20.178441 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise self.value Jul 26 13:34:20.178441 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 26 13:34:20.178441 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 26 13:34:20.178441 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 26 13:34:20.178441 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application session.commit() Jul 26 13:34:20.178441 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 26 13:34:20.178441 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 26 13:34:20.178441 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 26 13:34:20.180035 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application self._prepare_impl() Jul 26 13:34:20.180035 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 26 13:34:20.180035 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application self.session.flush() Jul 26 13:34:20.180035 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 26 13:34:20.180035 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application self._flush(objects) Jul 26 13:34:20.180035 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 26 13:34:20.180035 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 26 13:34:20.180035 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 26 13:34:20.180035 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application compat.raise_( Jul 26 13:34:20.180035 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 26 13:34:20.180035 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception Jul 26 13:34:20.180035 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 26 13:34:20.180035 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application flush_context.execute() Jul 26 13:34:20.180035 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 26 13:34:20.180035 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rec.execute(self) Jul 26 13:34:20.180035 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 26 13:34:20.180035 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 26 13:34:20.180937 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 26 13:34:20.180937 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 26 13:34:20.180937 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 26 13:34:20.180937 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 26 13:34:20.180937 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 26 13:34:20.180937 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 26 13:34:20.180937 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 26 13:34:20.180937 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 26 13:34:20.180937 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 26 13:34:20.180937 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 26 13:34:20.180937 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 26 13:34:20.180937 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 26 13:34:20.180937 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 26 13:34:20.180937 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 26 13:34:20.180937 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 26 13:34:20.180937 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception Jul 26 13:34:20.183044 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 26 13:34:20.183044 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 26 13:34:20.183044 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 26 13:34:20.183044 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 26 13:34:20.183044 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 26 13:34:20.183044 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application result = self._query(query) Jul 26 13:34:20.183044 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 26 13:34:20.183044 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application conn.query(q) Jul 26 13:34:20.183044 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 26 13:34:20.183044 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 26 13:34:20.183044 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 26 13:34:20.183044 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application result.read() Jul 26 13:34:20.183044 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 26 13:34:20.183044 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 26 13:34:20.183044 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 26 13:34:20.183044 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 26 13:34:20.185174 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 26 13:34:20.185174 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 26 13:34:20.185174 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 26 13:34:20.185174 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 26 13:34:20.185174 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 26 13:34:20.185174 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 26 13:34:20.185174 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [parameters: {'user_id': '81a5b870472249329622e6da96c814f1', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 26 13:34:20.185174 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 26 13:34:20.185174 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:34:20.185174 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 13:34:20.185174 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:34:20.185174 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:20.185174 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:34:20.185174 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:34:20.185174 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:34:20.187076 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:34:20.187076 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:34:20.187076 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:34:20.187076 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:34:20.187076 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:34:20.187076 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:34:20.187076 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:34:20.187076 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 26 13:34:20.187076 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 26 13:34:20.187076 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:34:20.187076 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:34:20.187076 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 13:34:20.187076 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 13:34:20.187076 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 13:34:20.187076 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 13:34:20.187076 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 26 13:34:20.187076 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 26 13:34:20.195811 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 26 13:34:20.195811 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 26 13:34:20.195811 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 26 13:34:20.195811 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 26 13:34:20.195811 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 26 13:34:20.195811 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:34:20.195811 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 478/963] 10.222.0.58 () {60 vars in 1131 bytes} [Tue Jul 26 13:34:19 2022] POST /identity/v3/users => generated 160 bytes in 197 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:34:20.196706 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812bb4a3-88bd-47c4-98c5-9887ac30c313 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:20.295607 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812bb4a3-88bd-47c4-98c5-9887ac30c313 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:20.331700 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a578b223-440e-4b57-8b76-03fde29fa071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:20.332356 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a578b223-440e-4b57-8b76-03fde29fa071 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:20.332637 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a578b223-440e-4b57-8b76-03fde29fa071 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:20.332910 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a578b223-440e-4b57-8b76-03fde29fa071 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:20.341176 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a578b223-440e-4b57-8b76-03fde29fa071 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:20.342646 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a578b223-440e-4b57-8b76-03fde29fa071 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:20.355165 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-a578b223-440e-4b57-8b76-03fde29fa071 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 26 13:34:20.355165 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:20.355165 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:34:20.355165 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:34:20.355165 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:34:20.355165 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:34:20.355165 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:34:20.355165 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:34:20.355165 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:34:20.355165 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:34:20.355165 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:34:20.355165 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:34:20.355165 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 13:34:20.355165 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 13:34:20.355165 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 13:34:20.355165 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 13:34:20.357346 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:34:20.357346 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:34:20.357346 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 13:34:20.357346 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 13:34:20.357346 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 13:34:20.357346 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 13:34:20.357346 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:34:20.357346 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:34:20.357346 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:34:20.357346 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:34:20.357346 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:34:20.357346 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:34:20.357346 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:34:20.357346 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:34:20.357346 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:34:20.357346 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:34:20.357346 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:34:20.357346 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:34:20.359620 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:34:20.359620 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:34:20.359620 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 13:34:20.359620 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 13:34:20.359620 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 13:34:20.359620 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 13:34:20.359620 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 13:34:20.359620 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 13:34:20.359620 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 26 13:34:20.359620 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:34:20.361522 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 486/964] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Jul 26 13:34:20 2022] GET /identity/v3/users/cinder => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:34:20.362928 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685b9cc0-0d7d-414a-9d06-5ae8cc0db45e None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:20.422404 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:20.469660 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685b9cc0-0d7d-414a-9d06-5ae8cc0db45e None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:20.506041 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd03f43-fef7-4f3c-bd37-4ffa146b4098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:20.506969 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd03f43-fef7-4f3c-bd37-4ffa146b4098 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:20.507407 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd03f43-fef7-4f3c-bd37-4ffa146b4098 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:20.507828 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd03f43-fef7-4f3c-bd37-4ffa146b4098 None admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:20.508766 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd03f43-fef7-4f3c-bd37-4ffa146b4098 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:20.509219 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd03f43-fef7-4f3c-bd37-4ffa146b4098 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:20.511011 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd03f43-fef7-4f3c-bd37-4ffa146b4098 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:20.555591 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 479/965] 10.222.0.58 () {56 vars in 1101 bytes} [Tue Jul 26 13:34:20 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:20.676817 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d739c9-de56-4e4d-8d79-aeb74443da46 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:20.677201 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d739c9-de56-4e4d-8d79-aeb74443da46 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:20.677632 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d739c9-de56-4e4d-8d79-aeb74443da46 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:20.679504 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 487/966] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:34:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:34:20.685379 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede33179-1734-491c-98d1-4c16210515a7 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:20.685711 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede33179-1734-491c-98d1-4c16210515a7 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:20.685970 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede33179-1734-491c-98d1-4c16210515a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:20.708138 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-ede33179-1734-491c-98d1-4c16210515a7 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:34:20.709728 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-ede33179-1734-491c-98d1-4c16210515a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:34:20.786666 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 480/967] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:34:20 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 13:34:20.810297 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d739c9-de56-4e4d-8d79-aeb74443da46 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:20.849266 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:20.861511 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd69a24-8a68-49e4-924f-8f053a2a3918 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:20.862043 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd69a24-8a68-49e4-924f-8f053a2a3918 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:20.862526 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd69a24-8a68-49e4-924f-8f053a2a3918 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:20.865150 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 481/968] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:34:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:34:20.873590 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7e7e55-58fd-46f8-92a6-bc0a19dc3030 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:20.874090 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7e7e55-58fd-46f8-92a6-bc0a19dc3030 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:20.874570 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7e7e55-58fd-46f8-92a6-bc0a19dc3030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:20.898448 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-4c7e7e55-58fd-46f8-92a6-bc0a19dc3030 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:34:20.899632 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-4c7e7e55-58fd-46f8-92a6-bc0a19dc3030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:34:20.919268 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d739c9-de56-4e4d-8d79-aeb74443da46 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:20.930943 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:20.954192 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950c5d0d-9f7b-4f5b-abce-94e896d49e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:20.954835 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950c5d0d-9f7b-4f5b-abce-94e896d49e47 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:20.955115 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950c5d0d-9f7b-4f5b-abce-94e896d49e47 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:20.955390 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950c5d0d-9f7b-4f5b-abce-94e896d49e47 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:20.956020 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950c5d0d-9f7b-4f5b-abce-94e896d49e47 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:20.957406 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950c5d0d-9f7b-4f5b-abce-94e896d49e47 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:20.966843 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-950c5d0d-9f7b-4f5b-abce-94e896d49e47 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 13:34:20.966843 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:20.966843 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 13:34:20.966843 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 13:34:20.966843 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 13:34:20.966843 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 13:34:20.966843 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 13:34:20.966843 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 13:34:20.966843 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 13:34:20.966843 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:34:20.966843 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 13:34:20.966843 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:34:20.966843 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:20.966843 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:34:20.966843 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:34:20.966843 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:34:20.966843 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:34:20.967880 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:34:20.967880 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:34:20.967880 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:34:20.967880 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:34:20.967880 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:34:20.967880 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:34:20.967880 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 13:34:20.967880 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 13:34:20.967880 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 13:34:20.967880 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 13:34:20.967880 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:34:20.967880 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:34:20.967880 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:34:20.967880 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:34:20.967880 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:34:20.967880 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:34:20.968727 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:34:20.968727 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:34:20.968727 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:34:20.968727 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:34:20.968727 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:34:20.968727 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:34:20.968727 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:34:20.968727 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:34:20.968727 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:34:20.968727 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:34:20.968727 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 13:34:20.968727 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 13:34:20.968727 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 13:34:20.968727 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:34:20.968727 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 488/969] 10.222.0.58 () {56 vars in 1126 bytes} [Tue Jul 26 13:34:20 2022] GET /identity/v3/services/object-store => generated 93 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:34:20.973919 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950c5d0d-9f7b-4f5b-abce-94e896d49e47 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:20.977092 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 482/970] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:34:20 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 13:34:21.000197 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7e7e55-58fd-46f8-92a6-bc0a19dc3030 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:21.082056 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:21.086204 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950c5d0d-9f7b-4f5b-abce-94e896d49e47 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:21.111722 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7e7e55-58fd-46f8-92a6-bc0a19dc3030 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:21.123779 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70435df5-eff7-4355-a14c-d8872c83ae08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:21.124411 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70435df5-eff7-4355-a14c-d8872c83ae08 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:21.125012 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70435df5-eff7-4355-a14c-d8872c83ae08 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:21.125295 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70435df5-eff7-4355-a14c-d8872c83ae08 None admin] PATH_INFO: `/v3/services` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:21.126171 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70435df5-eff7-4355-a14c-d8872c83ae08 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:21.126481 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70435df5-eff7-4355-a14c-d8872c83ae08 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:21.127850 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70435df5-eff7-4355-a14c-d8872c83ae08 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:21.133323 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 489/971] 10.222.0.58 () {56 vars in 1122 bytes} [Tue Jul 26 13:34:20 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:34:21.139557 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70435df5-eff7-4355-a14c-d8872c83ae08 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:21.149939 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f3fd68-8a3f-4376-a7ac-cad4513ea9d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:21.150590 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f3fd68-8a3f-4376-a7ac-cad4513ea9d9 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:21.150871 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f3fd68-8a3f-4376-a7ac-cad4513ea9d9 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:21.151140 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f3fd68-8a3f-4376-a7ac-cad4513ea9d9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:21.162023 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f3fd68-8a3f-4376-a7ac-cad4513ea9d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:34:21.166607 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-e8f3fd68-8a3f-4376-a7ac-cad4513ea9d9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 13:34:21.166607 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:21.166607 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:34:21.166607 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:34:21.166607 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:34:21.166607 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:34:21.166607 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:34:21.166607 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:34:21.166607 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:34:21.166607 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:34:21.166607 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:34:21.166607 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:34:21.166607 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:34:21.166607 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:34:21.166607 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:34:21.166607 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:34:21.167554 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:34:21.167554 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:34:21.167554 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:34:21.167554 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:34:21.167554 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:34:21.167554 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:34:21.167554 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:34:21.167554 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:34:21.167554 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:34:21.167554 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:34:21.167554 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:34:21.167554 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:34:21.167554 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:34:21.167554 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:34:21.167554 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:34:21.167554 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:34:21.169177 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:34:21.169177 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:34:21.169177 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:34:21.169177 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:34:21.169177 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:34:21.169177 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:34:21.169177 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:34:21.169177 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:34:21.169177 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:34:21.169177 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:34:21.169177 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 13:34:21.169177 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:34:21.169177 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 483/972] 10.222.0.58 () {56 vars in 1120 bytes} [Tue Jul 26 13:34:20 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 13:34:21.173794 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f3fd68-8a3f-4376-a7ac-cad4513ea9d9 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:21.205264 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:21.246593 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70435df5-eff7-4355-a14c-d8872c83ae08 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:21.278366 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f3fd68-8a3f-4376-a7ac-cad4513ea9d9 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:21.280165 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb6a870-d2c2-4e4e-939c-75bea676eeb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:21.280784 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb6a870-d2c2-4e4e-939c-75bea676eeb4 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:21.281185 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb6a870-d2c2-4e4e-939c-75bea676eeb4 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:21.281461 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb6a870-d2c2-4e4e-939c-75bea676eeb4 None admin] PATH_INFO: `/v3/services` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:21.282165 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb6a870-d2c2-4e4e-939c-75bea676eeb4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:21.282469 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb6a870-d2c2-4e4e-939c-75bea676eeb4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:21.283818 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb6a870-d2c2-4e4e-939c-75bea676eeb4 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:21.289296 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 490/973] 10.222.0.58 () {56 vars in 1122 bytes} [Tue Jul 26 13:34:21 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:21.290587 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a8cf00-5ec5-4313-a685-f82b6ee6f5a2 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:21.290931 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a8cf00-5ec5-4313-a685-f82b6ee6f5a2 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:21.291212 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a8cf00-5ec5-4313-a685-f82b6ee6f5a2 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:21.292598 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 491/974] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:34:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:34:21.297936 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32997de-9561-4c8d-9ea6-b229c6d0fe42 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:21.298370 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32997de-9561-4c8d-9ea6-b229c6d0fe42 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:21.298518 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32997de-9561-4c8d-9ea6-b229c6d0fe42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:21.316474 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-a32997de-9561-4c8d-9ea6-b229c6d0fe42 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:34:21.317687 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-a32997de-9561-4c8d-9ea6-b229c6d0fe42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:34:21.320745 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa87d3f9-8c5f-41e8-bc31-8114dfb8c4ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:21.321368 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa87d3f9-8c5f-41e8-bc31-8114dfb8c4ab None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:21.321748 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa87d3f9-8c5f-41e8-bc31-8114dfb8c4ab None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:21.322025 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa87d3f9-8c5f-41e8-bc31-8114dfb8c4ab None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:21.322731 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa87d3f9-8c5f-41e8-bc31-8114dfb8c4ab None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:21.323037 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa87d3f9-8c5f-41e8-bc31-8114dfb8c4ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:21.324397 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa87d3f9-8c5f-41e8-bc31-8114dfb8c4ab None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:21.337094 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 484/975] 10.222.0.58 () {56 vars in 1115 bytes} [Tue Jul 26 13:34:21 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:34:21.337094 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa87d3f9-8c5f-41e8-bc31-8114dfb8c4ab None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:21.337792 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:21.387058 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 492/976] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:34:21 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 13:34:21.394169 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32997de-9561-4c8d-9ea6-b229c6d0fe42 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:21.424381 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:21.437335 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa87d3f9-8c5f-41e8-bc31-8114dfb8c4ab None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:21.473326 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20139f8e-6f02-4272-afc4-15245be0f5f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:21.473877 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20139f8e-6f02-4272-afc4-15245be0f5f3 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:21.474160 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20139f8e-6f02-4272-afc4-15245be0f5f3 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:21.474435 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20139f8e-6f02-4272-afc4-15245be0f5f3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:21.475207 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20139f8e-6f02-4272-afc4-15245be0f5f3 None admin] RBAC: Adding query filter params (interface=admin, service_id=9fe7cfa7b96b411f8683612376ff5669, region_id=RegionOne) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:21.476037 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20139f8e-6f02-4272-afc4-15245be0f5f3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:21.477402 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20139f8e-6f02-4272-afc4-15245be0f5f3 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:21.482748 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 485/977] 10.222.0.58 () {56 vars in 1249 bytes} [Tue Jul 26 13:34:21 2022] GET /identity/v3/endpoints?service_id=9fe7cfa7b96b411f8683612376ff5669&interface=admin®ion_id=RegionOne => generated 193 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:21.483522 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20139f8e-6f02-4272-afc4-15245be0f5f3 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:21.492531 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32997de-9561-4c8d-9ea6-b229c6d0fe42 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:21.526501 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9d9e8d-d51e-421b-9158-d3f82dadb787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:21.527150 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9d9e8d-d51e-421b-9158-d3f82dadb787 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:21.527436 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9d9e8d-d51e-421b-9158-d3f82dadb787 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:21.527708 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9d9e8d-d51e-421b-9158-d3f82dadb787 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:21.537668 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9d9e8d-d51e-421b-9158-d3f82dadb787 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:21.539616 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9d9e8d-d51e-421b-9158-d3f82dadb787 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:21.554878 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-9e9d9e8d-d51e-421b-9158-d3f82dadb787 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:34:21.554878 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:21.554878 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:34:21.554878 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:34:21.554878 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:34:21.554878 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:34:21.554878 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:34:21.554878 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:34:21.554878 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:34:21.554878 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:34:21.554878 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:34:21.554878 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:34:21.554878 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 13:34:21.554878 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 13:34:21.554878 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 13:34:21.555957 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 13:34:21.555957 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:34:21.555957 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:34:21.555957 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:34:21.555957 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:34:21.555957 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:34:21.555957 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:34:21.555957 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:34:21.555957 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:34:21.555957 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:34:21.555957 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:34:21.555957 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:34:21.555957 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:34:21.555957 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:34:21.555957 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:34:21.555957 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:34:21.555957 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:34:21.556957 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 13:34:21.556957 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 13:34:21.556957 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:34:21.556957 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:34:21.556957 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 493/978] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:34:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:34:21.557239 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9d9e8d-d51e-421b-9158-d3f82dadb787 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:21.568092 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:21.588470 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20139f8e-6f02-4272-afc4-15245be0f5f3 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:21.622945 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52407df-24f6-4a43-b304-8cd2d9578895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:21.623569 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52407df-24f6-4a43-b304-8cd2d9578895 None admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:21.623963 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52407df-24f6-4a43-b304-8cd2d9578895 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:21.624132 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52407df-24f6-4a43-b304-8cd2d9578895 None admin] PATH_INFO: `/v3/services` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:21.624821 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52407df-24f6-4a43-b304-8cd2d9578895 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:21.626217 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52407df-24f6-4a43-b304-8cd2d9578895 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:21.640954 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 486/979] 10.222.0.58 () {60 vars in 1141 bytes} [Tue Jul 26 13:34:21 2022] POST /identity/v3/services => generated 242 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:34:21.653743 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52407df-24f6-4a43-b304-8cd2d9578895 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:21.658397 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:21.663595 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9d9e8d-d51e-421b-9158-d3f82dadb787 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:21.715391 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3ecdaf-01dd-4d95-946a-bd9df6b9f89c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:21.716162 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3ecdaf-01dd-4d95-946a-bd9df6b9f89c None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:21.716509 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3ecdaf-01dd-4d95-946a-bd9df6b9f89c None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:21.717339 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3ecdaf-01dd-4d95-946a-bd9df6b9f89c None admin] PATH_INFO: `/v3/services` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:21.717639 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3ecdaf-01dd-4d95-946a-bd9df6b9f89c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:21.719114 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3ecdaf-01dd-4d95-946a-bd9df6b9f89c None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:21.737616 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 494/980] 10.222.0.58 () {56 vars in 1087 bytes} [Tue Jul 26 13:34:21 2022] GET /identity/v3/services => generated 1727 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:34:21.782672 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52407df-24f6-4a43-b304-8cd2d9578895 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:21.826066 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5456289b-a696-41d2-9cbc-42d9351e4bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:21.827625 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5456289b-a696-41d2-9cbc-42d9351e4bf5 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:21.827625 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5456289b-a696-41d2-9cbc-42d9351e4bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:21.827625 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5456289b-a696-41d2-9cbc-42d9351e4bf5 None admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:21.828778 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5456289b-a696-41d2-9cbc-42d9351e4bf5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:21.828778 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5456289b-a696-41d2-9cbc-42d9351e4bf5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:21.830261 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5456289b-a696-41d2-9cbc-42d9351e4bf5 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:21.844552 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 487/981] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Jul 26 13:34:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:21.851086 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3ecdaf-01dd-4d95-946a-bd9df6b9f89c None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:21.923552 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:21.960414 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3ecdaf-01dd-4d95-946a-bd9df6b9f89c None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:21.996034 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d874a59-71a4-4865-a5dc-27b3604a4c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:21.996608 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d874a59-71a4-4865-a5dc-27b3604a4c55 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:21.996895 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d874a59-71a4-4865-a5dc-27b3604a4c55 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:21.997175 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d874a59-71a4-4865-a5dc-27b3604a4c55 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:22.005252 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d874a59-71a4-4865-a5dc-27b3604a4c55 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:22.006775 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d874a59-71a4-4865-a5dc-27b3604a4c55 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:22.018512 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-6d874a59-71a4-4865-a5dc-27b3604a4c55 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 13:34:22.018512 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:22.018512 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:34:22.018512 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:34:22.018512 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:34:22.018512 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:34:22.018512 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:34:22.018512 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:34:22.018512 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:34:22.018512 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:34:22.018512 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:34:22.018512 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:34:22.018512 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 13:34:22.018512 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 13:34:22.018512 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 13:34:22.018512 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 13:34:22.019461 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:34:22.019461 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:34:22.019461 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 13:34:22.019461 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 13:34:22.019461 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 13:34:22.019461 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 13:34:22.019461 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:34:22.019461 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:34:22.019461 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:34:22.019461 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:34:22.019461 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:34:22.019461 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:34:22.019461 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:34:22.019461 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:34:22.019461 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:34:22.019461 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:34:22.019461 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:34:22.019461 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:34:22.020439 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:34:22.020439 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:34:22.020439 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 13:34:22.020439 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 13:34:22.020439 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 13:34:22.020439 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 13:34:22.020439 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 13:34:22.020439 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 13:34:22.020439 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 13:34:22.020439 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:34:22.020439 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 495/982] 10.222.0.58 () {56 vars in 1117 bytes} [Tue Jul 26 13:34:21 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:34:22.026256 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5456289b-a696-41d2-9cbc-42d9351e4bf5 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:22.101817 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:22.133979 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5456289b-a696-41d2-9cbc-42d9351e4bf5 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:22.170365 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facbac71-6827-409c-adbc-f611b046e9e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:22.170999 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facbac71-6827-409c-adbc-f611b046e9e4 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:22.171279 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facbac71-6827-409c-adbc-f611b046e9e4 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:22.171806 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facbac71-6827-409c-adbc-f611b046e9e4 None admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:22.172606 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facbac71-6827-409c-adbc-f611b046e9e4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:22.172915 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facbac71-6827-409c-adbc-f611b046e9e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:22.174532 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facbac71-6827-409c-adbc-f611b046e9e4 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:22.197882 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 488/983] 10.222.0.58 () {56 vars in 1149 bytes} [Tue Jul 26 13:34:22 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:22.199013 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d874a59-71a4-4865-a5dc-27b3604a4c55 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:22.308193 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d874a59-71a4-4865-a5dc-27b3604a4c55 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:22.335522 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:22.344354 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60947fb1-c3e3-4055-8543-2653db3e5f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:22.344980 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60947fb1-c3e3-4055-8543-2653db3e5f55 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:22.345260 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60947fb1-c3e3-4055-8543-2653db3e5f55 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:22.345569 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60947fb1-c3e3-4055-8543-2653db3e5f55 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:22.412987 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60947fb1-c3e3-4055-8543-2653db3e5f55 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:22.414462 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60947fb1-c3e3-4055-8543-2653db3e5f55 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:22.518528 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 496/984] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Jul 26 13:34:22 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 13:34:22.519703 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facbac71-6827-409c-adbc-f611b046e9e4 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:22.618526 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facbac71-6827-409c-adbc-f611b046e9e4 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:22.647646 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:22.654819 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb95e1c-4416-4652-bf6b-b1c609948735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:22.655552 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb95e1c-4416-4652-bf6b-b1c609948735 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:22.655903 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb95e1c-4416-4652-bf6b-b1c609948735 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:22.656246 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb95e1c-4416-4652-bf6b-b1c609948735 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:22.668487 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb95e1c-4416-4652-bf6b-b1c609948735 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:22.670873 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb95e1c-4416-4652-bf6b-b1c609948735 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:22.687508 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-acb95e1c-4416-4652-bf6b-b1c609948735 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:34:22.687508 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:22.687508 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:34:22.687508 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:34:22.687508 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:34:22.687508 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:34:22.687508 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:34:22.687508 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:34:22.687508 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:34:22.687508 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:34:22.687508 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:34:22.687508 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:34:22.687508 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 13:34:22.687508 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 13:34:22.687508 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 13:34:22.688633 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 13:34:22.688633 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:34:22.688633 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:34:22.688633 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:34:22.688633 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:34:22.688633 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:34:22.688633 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:34:22.688633 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:34:22.688633 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:34:22.688633 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:34:22.688633 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:34:22.688633 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:34:22.688633 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:34:22.688633 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:34:22.688633 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:34:22.688633 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:34:22.688633 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:34:22.689744 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 13:34:22.689744 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 13:34:22.689744 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:34:22.689744 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:34:22.695133 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 489/985] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:34:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:34:22.696492 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60947fb1-c3e3-4055-8543-2653db3e5f55 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:22.731013 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:22.827088 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60947fb1-c3e3-4055-8543-2653db3e5f55 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:22.881117 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1385da42-64a4-4539-a21a-e1ff9abd22ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:22.882058 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1385da42-64a4-4539-a21a-e1ff9abd22ca None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:22.882441 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1385da42-64a4-4539-a21a-e1ff9abd22ca None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:22.882823 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1385da42-64a4-4539-a21a-e1ff9abd22ca None admin] PATH_INFO: `/v3/domains` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:22.883625 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1385da42-64a4-4539-a21a-e1ff9abd22ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:22.884151 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1385da42-64a4-4539-a21a-e1ff9abd22ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:22.885652 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1385da42-64a4-4539-a21a-e1ff9abd22ca None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:22.902744 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 497/986] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Jul 26 13:34:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:34:22.904109 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb95e1c-4416-4652-bf6b-b1c609948735 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:23.013336 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb95e1c-4416-4652-bf6b-b1c609948735 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:23.059007 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314cdc06-e420-4ca9-ac50-291be220fd29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:23.059683 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314cdc06-e420-4ca9-ac50-291be220fd29 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:23.059931 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314cdc06-e420-4ca9-ac50-291be220fd29 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:23.060206 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314cdc06-e420-4ca9-ac50-291be220fd29 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:23.067393 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314cdc06-e420-4ca9-ac50-291be220fd29 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:23.068851 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314cdc06-e420-4ca9-ac50-291be220fd29 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:23.081326 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-314cdc06-e420-4ca9-ac50-291be220fd29 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 13:34:23.081326 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:23.081326 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:34:23.081326 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:34:23.081326 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:34:23.081326 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:34:23.081326 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:34:23.081326 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:34:23.081326 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:34:23.081326 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:34:23.081326 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:34:23.081326 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:34:23.081326 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 13:34:23.081326 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 13:34:23.081326 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 13:34:23.082418 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 13:34:23.082418 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:34:23.082418 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:34:23.082418 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:34:23.082418 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:34:23.082418 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:34:23.082418 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:34:23.082418 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:34:23.082418 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:34:23.082418 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:34:23.082418 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:34:23.082418 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:34:23.082418 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:34:23.082418 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:34:23.082418 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:34:23.082418 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:34:23.082418 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:34:23.088296 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 13:34:23.088296 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 13:34:23.088296 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 13:34:23.088296 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 13:34:23.088296 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 13:34:23.088296 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 13:34:23.088296 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 13:34:23.088296 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:34:23.088296 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 490/987] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Jul 26 13:34:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:34:23.088781 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1385da42-64a4-4539-a21a-e1ff9abd22ca None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:23.182091 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:23.195246 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1385da42-64a4-4539-a21a-e1ff9abd22ca None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:23.229340 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43ec2b6-2e3f-47de-806c-cfe490d73972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:23.230075 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43ec2b6-2e3f-47de-806c-cfe490d73972 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:23.230361 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43ec2b6-2e3f-47de-806c-cfe490d73972 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:23.230637 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43ec2b6-2e3f-47de-806c-cfe490d73972 None admin] PATH_INFO: `/v3/projects` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:23.231425 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43ec2b6-2e3f-47de-806c-cfe490d73972 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:23.231739 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43ec2b6-2e3f-47de-806c-cfe490d73972 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:23.233154 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43ec2b6-2e3f-47de-806c-cfe490d73972 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:23.249522 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 498/988] 10.222.0.58 () {56 vars in 1148 bytes} [Tue Jul 26 13:34:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:23.251924 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314cdc06-e420-4ca9-ac50-291be220fd29 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:23.296789 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:23.354664 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314cdc06-e420-4ca9-ac50-291be220fd29 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:23.387914 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3faa6d-0447-4688-9393-0f56f53bd7a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:23.388538 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3faa6d-0447-4688-9393-0f56f53bd7a2 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:23.388819 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3faa6d-0447-4688-9393-0f56f53bd7a2 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:23.389094 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3faa6d-0447-4688-9393-0f56f53bd7a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:23.389913 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3faa6d-0447-4688-9393-0f56f53bd7a2 None admin] RBAC: Adding query filter params (role.id=ac11ff5f40194d7d98f81a5429aa3826, scope.project.id=1e8bb4385c204210a1bc77b1f0d654f3, user.id=b64bd7b183d24d8c9037b261c7884810) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:23.390224 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3faa6d-0447-4688-9393-0f56f53bd7a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:23.391640 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3faa6d-0447-4688-9393-0f56f53bd7a2 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:23.405731 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 491/989] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Jul 26 13:34:23 2022] GET /identity/v3/role_assignments?user.id=b64bd7b183d24d8c9037b261c7884810&scope.project.id=1e8bb4385c204210a1bc77b1f0d654f3&role.id=ac11ff5f40194d7d98f81a5429aa3826 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:34:23.702153 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3b7787-7fec-4102-ad0c-92cf1c0459f6 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:23.702679 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3b7787-7fec-4102-ad0c-92cf1c0459f6 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:23.702795 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3b7787-7fec-4102-ad0c-92cf1c0459f6 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:23.704230 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 499/990] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:34:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:34:23.709132 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c36338b-163a-44ce-b4f8-220c48dc7b0a None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:23.709425 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c36338b-163a-44ce-b4f8-220c48dc7b0a None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:23.710277 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c36338b-163a-44ce-b4f8-220c48dc7b0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:23.728928 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-9c36338b-163a-44ce-b4f8-220c48dc7b0a None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:34:23.730369 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-9c36338b-163a-44ce-b4f8-220c48dc7b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:34:23.802988 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 492/991] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:34:23 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 13:34:23.812430 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1511cc5-317c-4e1d-8daa-3b08a436195a None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:23.812718 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1511cc5-317c-4e1d-8daa-3b08a436195a None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:23.812995 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1511cc5-317c-4e1d-8daa-3b08a436195a None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:23.814499 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 500/992] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:34:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:34:23.819967 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef5ff74-9771-439a-86bf-f3e55f97bf0d None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:23.820259 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef5ff74-9771-439a-86bf-f3e55f97bf0d None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:23.820532 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef5ff74-9771-439a-86bf-f3e55f97bf0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:23.825901 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1511cc5-317c-4e1d-8daa-3b08a436195a None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:23.839332 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-6ef5ff74-9771-439a-86bf-f3e55f97bf0d None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:34:23.840527 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-6ef5ff74-9771-439a-86bf-f3e55f97bf0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:34:23.879517 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:23.928308 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 493/993] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:34:23 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 13:34:23.935086 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1511cc5-317c-4e1d-8daa-3b08a436195a None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:23.951893 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef5ff74-9771-439a-86bf-f3e55f97bf0d None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:23.971066 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d976a05-fd64-49dc-9dd5-cf84d2ae2b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:23.971614 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d976a05-fd64-49dc-9dd5-cf84d2ae2b0b None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:23.971960 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d976a05-fd64-49dc-9dd5-cf84d2ae2b0b None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:23.972246 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d976a05-fd64-49dc-9dd5-cf84d2ae2b0b None admin] PATH_INFO: `/v3/services/compute` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:23.972866 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d976a05-fd64-49dc-9dd5-cf84d2ae2b0b None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:23.974318 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d976a05-fd64-49dc-9dd5-cf84d2ae2b0b None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:23.983923 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-5d976a05-fd64-49dc-9dd5-cf84d2ae2b0b None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 26 13:34:23.983923 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:23.983923 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 13:34:23.983923 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 13:34:23.983923 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 13:34:23.983923 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 13:34:23.983923 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 13:34:23.983923 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 13:34:23.983923 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 26 13:34:23.983923 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:34:23.983923 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 13:34:23.983923 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:34:23.983923 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:23.983923 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:34:23.983923 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:34:23.983923 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:34:23.983923 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:34:23.984928 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:34:23.984928 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:34:23.984928 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:34:23.984928 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:34:23.984928 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:34:23.984928 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:34:23.984928 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 13:34:23.984928 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 13:34:23.984928 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 13:34:23.984928 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 13:34:23.984928 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:34:23.984928 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:34:23.984928 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:34:23.984928 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:34:23.984928 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:34:23.984928 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:34:23.985819 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:34:23.985819 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:34:23.985819 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:34:23.985819 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:34:23.985819 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:34:23.985819 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:34:23.985819 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:34:23.985819 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:34:23.985819 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:34:23.985819 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:34:23.985819 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 13:34:23.985819 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 13:34:23.985819 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 26 13:34:23.985819 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:34:23.985819 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 501/994] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Jul 26 13:34:23 2022] GET /identity/v3/services/compute => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 13:34:23.991619 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d976a05-fd64-49dc-9dd5-cf84d2ae2b0b None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:24.007302 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:24.071316 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef5ff74-9771-439a-86bf-f3e55f97bf0d None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:24.107827 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d976a05-fd64-49dc-9dd5-cf84d2ae2b0b None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:24.111857 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2ae29b-c691-4a10-be1c-ba5522c53b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:24.112517 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2ae29b-c691-4a10-be1c-ba5522c53b96 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:24.112801 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2ae29b-c691-4a10-be1c-ba5522c53b96 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:24.113077 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2ae29b-c691-4a10-be1c-ba5522c53b96 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:24.113760 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2ae29b-c691-4a10-be1c-ba5522c53b96 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:24.115141 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2ae29b-c691-4a10-be1c-ba5522c53b96 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:24.129026 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-0c2ae29b-c691-4a10-be1c-ba5522c53b96 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 13:34:24.129026 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:24.129026 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 13:34:24.129026 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 13:34:24.129026 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 13:34:24.129026 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 13:34:24.129026 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 13:34:24.129026 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 13:34:24.129026 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 13:34:24.129026 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:34:24.129026 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 13:34:24.129026 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:34:24.129026 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:24.129026 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:34:24.129026 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:34:24.129026 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:34:24.129026 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:34:24.130166 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:34:24.130166 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:34:24.130166 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:34:24.130166 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:34:24.130166 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:34:24.130166 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:34:24.130166 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 13:34:24.130166 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 13:34:24.130166 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 13:34:24.130166 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 13:34:24.130166 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:34:24.130166 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:34:24.130166 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:34:24.130166 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:34:24.130166 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:34:24.130166 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:34:24.131024 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:34:24.131024 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:34:24.131024 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:34:24.131024 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:34:24.131024 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:34:24.131024 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:34:24.131024 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:34:24.131024 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:34:24.131024 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:34:24.131024 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:34:24.131024 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 13:34:24.131024 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 13:34:24.131024 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 13:34:24.131024 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:34:24.131024 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 494/995] 10.222.0.58 () {56 vars in 1126 bytes} [Tue Jul 26 13:34:23 2022] GET /identity/v3/services/object-store => generated 93 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 13:34:24.137249 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2ae29b-c691-4a10-be1c-ba5522c53b96 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:24.158983 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51c9687-1b50-4008-9e74-1dfd8259bc68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:24.160198 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51c9687-1b50-4008-9e74-1dfd8259bc68 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:24.160876 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51c9687-1b50-4008-9e74-1dfd8259bc68 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:24.161671 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51c9687-1b50-4008-9e74-1dfd8259bc68 None admin] PATH_INFO: `/v3/services` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:24.164385 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51c9687-1b50-4008-9e74-1dfd8259bc68 None admin] RBAC: Adding query filter params (name=compute) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:24.164929 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51c9687-1b50-4008-9e74-1dfd8259bc68 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:24.166699 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51c9687-1b50-4008-9e74-1dfd8259bc68 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:24.171530 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:24.172751 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 502/996] 10.222.0.58 () {56 vars in 1112 bytes} [Tue Jul 26 13:34:23 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:34:24.179484 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51c9687-1b50-4008-9e74-1dfd8259bc68 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:24.253812 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2ae29b-c691-4a10-be1c-ba5522c53b96 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:24.266427 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:24.299358 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51c9687-1b50-4008-9e74-1dfd8259bc68 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:24.304593 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f059847-bd26-4648-8d6e-d71596146337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:24.305219 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f059847-bd26-4648-8d6e-d71596146337 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:24.305534 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f059847-bd26-4648-8d6e-d71596146337 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:24.305813 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f059847-bd26-4648-8d6e-d71596146337 None admin] PATH_INFO: `/v3/services` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:24.306499 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f059847-bd26-4648-8d6e-d71596146337 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:24.306810 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f059847-bd26-4648-8d6e-d71596146337 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:24.308180 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f059847-bd26-4648-8d6e-d71596146337 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:24.313377 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 495/997] 10.222.0.58 () {56 vars in 1122 bytes} [Tue Jul 26 13:34:24 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:34:24.318899 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f059847-bd26-4648-8d6e-d71596146337 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:24.340920 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8170e568-b951-428c-b995-e6f7503cb2fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:24.341679 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8170e568-b951-428c-b995-e6f7503cb2fa None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:24.341966 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8170e568-b951-428c-b995-e6f7503cb2fa None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:24.342251 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8170e568-b951-428c-b995-e6f7503cb2fa None admin] PATH_INFO: `/v3/services` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:24.342938 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8170e568-b951-428c-b995-e6f7503cb2fa None admin] RBAC: Adding query filter params (type=compute) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:24.343251 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8170e568-b951-428c-b995-e6f7503cb2fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:24.344614 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8170e568-b951-428c-b995-e6f7503cb2fa None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:24.355577 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 503/998] 10.222.0.58 () {56 vars in 1112 bytes} [Tue Jul 26 13:34:24 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:34:24.356987 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8170e568-b951-428c-b995-e6f7503cb2fa None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:24.418444 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:24.426867 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f059847-bd26-4648-8d6e-d71596146337 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:24.462847 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71f736a-a723-443b-bdb0-97f33acf0e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:24.463476 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71f736a-a723-443b-bdb0-97f33acf0e44 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:24.463764 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71f736a-a723-443b-bdb0-97f33acf0e44 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:24.464048 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71f736a-a723-443b-bdb0-97f33acf0e44 None admin] PATH_INFO: `/v3/services` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:24.464722 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71f736a-a723-443b-bdb0-97f33acf0e44 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:24.465031 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71f736a-a723-443b-bdb0-97f33acf0e44 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:24.466538 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71f736a-a723-443b-bdb0-97f33acf0e44 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:24.471671 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 496/999] 10.222.0.58 () {56 vars in 1122 bytes} [Tue Jul 26 13:34:24 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:34:24.473862 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8170e568-b951-428c-b995-e6f7503cb2fa None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:24.477699 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71f736a-a723-443b-bdb0-97f33acf0e44 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:24.513519 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22a6520-9655-49bc-9695-6df88fe14165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:24.514199 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a6520-9655-49bc-9695-6df88fe14165 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:24.514368 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a6520-9655-49bc-9695-6df88fe14165 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:24.514611 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a6520-9655-49bc-9695-6df88fe14165 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:24.515377 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22a6520-9655-49bc-9695-6df88fe14165 None admin] RBAC: Adding query filter params (interface=public, service_id=8ceb4da1dc0e455baa1641a55e7af777, region_id=RegionOne) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:24.515691 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22a6520-9655-49bc-9695-6df88fe14165 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:24.517066 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22a6520-9655-49bc-9695-6df88fe14165 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:24.527452 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 504/1000] 10.222.0.58 () {56 vars in 1251 bytes} [Tue Jul 26 13:34:24 2022] GET /identity/v3/endpoints?service_id=8ceb4da1dc0e455baa1641a55e7af777&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:24.528394 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22a6520-9655-49bc-9695-6df88fe14165 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:24.533135 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:24.587915 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71f736a-a723-443b-bdb0-97f33acf0e44 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:24.625112 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a78238-cde1-4aba-8789-6ea8d601c509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:24.625777 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a78238-cde1-4aba-8789-6ea8d601c509 None admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:24.626070 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a78238-cde1-4aba-8789-6ea8d601c509 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:24.626343 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a78238-cde1-4aba-8789-6ea8d601c509 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:24.627058 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a78238-cde1-4aba-8789-6ea8d601c509 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:24.628422 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a78238-cde1-4aba-8789-6ea8d601c509 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:24.645297 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 497/1001] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 13:34:24 2022] POST /identity/v3/endpoints => generated 325 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 13:34:24.646392 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22a6520-9655-49bc-9695-6df88fe14165 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:24.699742 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cf8314-2ae3-42ff-984e-dd705664f83b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:24.700392 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cf8314-2ae3-42ff-984e-dd705664f83b None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:24.700673 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cf8314-2ae3-42ff-984e-dd705664f83b None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:24.700947 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cf8314-2ae3-42ff-984e-dd705664f83b None admin] PATH_INFO: `/v3/services` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:24.701649 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cf8314-2ae3-42ff-984e-dd705664f83b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:24.703058 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cf8314-2ae3-42ff-984e-dd705664f83b None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:24.717823 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 505/1002] 10.222.0.58 () {56 vars in 1087 bytes} [Tue Jul 26 13:34:24 2022] GET /identity/v3/services => generated 1727 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:34:25.501394 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97202524-88b1-44b8-b863-b876d5f1f680 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:25.501948 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97202524-88b1-44b8-b863-b876d5f1f680 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:25.501948 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97202524-88b1-44b8-b863-b876d5f1f680 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:25.503484 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 498/1003] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:34:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:34:25.509382 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4ab856-00a8-4f1d-b060-171064d39d9e None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:25.509738 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4ab856-00a8-4f1d-b060-171064d39d9e None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:25.510013 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4ab856-00a8-4f1d-b060-171064d39d9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:25.528654 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-cd4ab856-00a8-4f1d-b060-171064d39d9e None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:34:25.529872 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-cd4ab856-00a8-4f1d-b060-171064d39d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:34:25.592223 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:25.601535 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 506/1004] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:34:25 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 13:34:25.623383 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97202524-88b1-44b8-b863-b876d5f1f680 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:25.644140 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:25.732979 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97202524-88b1-44b8-b863-b876d5f1f680 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:25.767984 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa79a359-1839-46b2-9a3d-38efe483384a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:25.768552 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa79a359-1839-46b2-9a3d-38efe483384a None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:25.768833 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa79a359-1839-46b2-9a3d-38efe483384a None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:25.769157 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa79a359-1839-46b2-9a3d-38efe483384a None admin] PATH_INFO: `/v3/services/image` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:25.769814 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa79a359-1839-46b2-9a3d-38efe483384a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:25.771233 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa79a359-1839-46b2-9a3d-38efe483384a None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:25.780335 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-aa79a359-1839-46b2-9a3d-38efe483384a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 26 13:34:25.780335 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:25.780335 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 13:34:25.780335 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 13:34:25.780335 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 13:34:25.780335 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 13:34:25.780335 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 13:34:25.780335 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 13:34:25.780335 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 26 13:34:25.780335 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:34:25.780335 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 13:34:25.780335 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:34:25.780335 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:25.780335 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:34:25.780335 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:34:25.780335 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:34:25.780335 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:34:25.781329 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:34:25.781329 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:34:25.781329 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:34:25.781329 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:34:25.781329 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:34:25.781329 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:34:25.781329 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 13:34:25.781329 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 13:34:25.781329 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 13:34:25.781329 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 13:34:25.781329 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:34:25.781329 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:34:25.781329 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:34:25.781329 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:34:25.781329 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:34:25.781329 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:34:25.782622 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:34:25.782622 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:34:25.782622 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:34:25.782622 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:34:25.782622 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:34:25.782622 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:34:25.782622 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:34:25.782622 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:34:25.782622 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:34:25.782622 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:34:25.782622 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 13:34:25.782622 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 13:34:25.782622 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 26 13:34:25.782622 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:34:25.786488 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 499/1005] 10.222.0.58 () {56 vars in 1105 bytes} [Tue Jul 26 13:34:25 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:34:25.788348 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4ab856-00a8-4f1d-b060-171064d39d9e None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:25.896339 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4ab856-00a8-4f1d-b060-171064d39d9e None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:25.933580 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b8311e-31c6-438a-84a9-9420bd5e6d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:25.934205 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b8311e-31c6-438a-84a9-9420bd5e6d62 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:25.934483 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b8311e-31c6-438a-84a9-9420bd5e6d62 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:25.934754 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b8311e-31c6-438a-84a9-9420bd5e6d62 None admin] PATH_INFO: `/v3/services` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:25.935473 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b8311e-31c6-438a-84a9-9420bd5e6d62 None admin] RBAC: Adding query filter params (name=image) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:25.935778 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b8311e-31c6-438a-84a9-9420bd5e6d62 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:25.937289 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b8311e-31c6-438a-84a9-9420bd5e6d62 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:25.947011 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 507/1006] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:34:25 2022] GET /identity/v3/services?name=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:25.948634 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa79a359-1839-46b2-9a3d-38efe483384a None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:26.040029 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:26.059640 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa79a359-1839-46b2-9a3d-38efe483384a None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:26.098206 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5988a97f-95e2-4495-9ece-4141dfe7a343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:26.098837 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5988a97f-95e2-4495-9ece-4141dfe7a343 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:26.099113 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5988a97f-95e2-4495-9ece-4141dfe7a343 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:26.099380 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5988a97f-95e2-4495-9ece-4141dfe7a343 None admin] PATH_INFO: `/v3/services` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:26.100058 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5988a97f-95e2-4495-9ece-4141dfe7a343 None admin] RBAC: Adding query filter params (type=image) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:26.100869 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5988a97f-95e2-4495-9ece-4141dfe7a343 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:26.102260 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5988a97f-95e2-4495-9ece-4141dfe7a343 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:26.113124 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 500/1007] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:34:25 2022] GET /identity/v3/services?type=image => generated 122 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:34:26.776918 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8429979a-1868-4d93-b7e5-74c64c6fa6a3 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:26.777301 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8429979a-1868-4d93-b7e5-74c64c6fa6a3 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:26.777502 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8429979a-1868-4d93-b7e5-74c64c6fa6a3 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:26.779021 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 508/1008] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:34:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:34:26.784116 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc64b45c-901f-4d71-a99f-0a8b11382110 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:26.784580 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc64b45c-901f-4d71-a99f-0a8b11382110 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:26.784658 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc64b45c-901f-4d71-a99f-0a8b11382110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:26.803641 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-cc64b45c-901f-4d71-a99f-0a8b11382110 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:34:26.804818 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-cc64b45c-901f-4d71-a99f-0a8b11382110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:34:26.839143 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f40c01-74a9-476c-8166-7543ee49edfe None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:26.839687 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f40c01-74a9-476c-8166-7543ee49edfe None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:26.839687 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f40c01-74a9-476c-8166-7543ee49edfe None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:26.841022 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 509/1009] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:34:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 13:34:26.846154 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ced2b2-e3a5-4d4e-8b6c-cbc3c5b2e1a4 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:26.846433 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ced2b2-e3a5-4d4e-8b6c-cbc3c5b2e1a4 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:26.846704 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ced2b2-e3a5-4d4e-8b6c-cbc3c5b2e1a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:26.865371 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-b8ced2b2-e3a5-4d4e-8b6c-cbc3c5b2e1a4 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:34:26.866627 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-b8ced2b2-e3a5-4d4e-8b6c-cbc3c5b2e1a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:34:26.893621 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 501/1010] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:34:26 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 13:34:26.907798 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc64b45c-901f-4d71-a99f-0a8b11382110 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:26.925560 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:26.942034 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 510/1011] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:34:26 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 13:34:26.965106 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ced2b2-e3a5-4d4e-8b6c-cbc3c5b2e1a4 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:26.986570 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:27.011666 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc64b45c-901f-4d71-a99f-0a8b11382110 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:27.052390 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331278d3-1fc5-4c09-bbd3-38c026c25e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:27.053040 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331278d3-1fc5-4c09-bbd3-38c026c25e24 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:27.053357 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331278d3-1fc5-4c09-bbd3-38c026c25e24 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:27.053665 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331278d3-1fc5-4c09-bbd3-38c026c25e24 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:27.054825 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331278d3-1fc5-4c09-bbd3-38c026c25e24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:27.056266 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331278d3-1fc5-4c09-bbd3-38c026c25e24 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:27.058070 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 502/1012] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:34:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:34:27.063719 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331278d3-1fc5-4c09-bbd3-38c026c25e24 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:27.080430 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ced2b2-e3a5-4d4e-8b6c-cbc3c5b2e1a4 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:27.118832 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261d8169-3677-436a-817b-e5ed863e953a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:27.119477 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261d8169-3677-436a-817b-e5ed863e953a None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:27.119751 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261d8169-3677-436a-817b-e5ed863e953a None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:27.120047 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261d8169-3677-436a-817b-e5ed863e953a None admin] PATH_INFO: `/v3/services/compute` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:27.120672 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261d8169-3677-436a-817b-e5ed863e953a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:27.122062 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261d8169-3677-436a-817b-e5ed863e953a None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:27.131029 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-261d8169-3677-436a-817b-e5ed863e953a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 26 13:34:27.131029 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:27.131029 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 13:34:27.131029 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 13:34:27.131029 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 13:34:27.131029 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 13:34:27.131029 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 13:34:27.131029 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 13:34:27.131029 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 26 13:34:27.131029 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:34:27.131029 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 13:34:27.131029 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:34:27.131029 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:27.131029 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:34:27.131029 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:34:27.131029 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:34:27.131029 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:34:27.132144 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:34:27.132144 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:34:27.132144 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:34:27.132144 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:34:27.132144 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:34:27.132144 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:34:27.132144 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 13:34:27.132144 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 13:34:27.132144 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 13:34:27.132144 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 13:34:27.132144 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:34:27.132144 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:34:27.132144 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:34:27.132144 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:34:27.132144 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:34:27.132144 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:34:27.133047 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:34:27.133047 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:34:27.133047 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:34:27.133047 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:34:27.133047 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:34:27.133047 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:34:27.133047 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:34:27.133047 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:34:27.133047 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:34:27.133047 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:34:27.133047 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 13:34:27.133047 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 13:34:27.133047 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 26 13:34:27.133047 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:34:27.137304 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 511/1013] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Jul 26 13:34:26 2022] GET /identity/v3/services/compute => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 13:34:27.138048 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261d8169-3677-436a-817b-e5ed863e953a None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:27.173542 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331278d3-1fc5-4c09-bbd3-38c026c25e24 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:27.193353 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:27.211010 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96abe07-f655-48cb-8205-86da6d5aabe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:27.211719 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96abe07-f655-48cb-8205-86da6d5aabe5 None admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:27.211920 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96abe07-f655-48cb-8205-86da6d5aabe5 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:27.212194 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96abe07-f655-48cb-8205-86da6d5aabe5 None admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:27.212876 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96abe07-f655-48cb-8205-86da6d5aabe5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:27.214338 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96abe07-f655-48cb-8205-86da6d5aabe5 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:27.232192 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:27.246299 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 503/1014] 10.222.0.58 () {60 vars in 1141 bytes} [Tue Jul 26 13:34:27 2022] POST /identity/v3/projects => generated 311 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:34:27.250963 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261d8169-3677-436a-817b-e5ed863e953a None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:27.291576 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8350e86-a9e3-458b-a9ef-51bc3c4e9b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:27.292182 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8350e86-a9e3-458b-a9ef-51bc3c4e9b14 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:27.292466 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8350e86-a9e3-458b-a9ef-51bc3c4e9b14 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:27.292755 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8350e86-a9e3-458b-a9ef-51bc3c4e9b14 None admin] PATH_INFO: `/v3/services` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:27.293525 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8350e86-a9e3-458b-a9ef-51bc3c4e9b14 None admin] RBAC: Adding query filter params (name=compute) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:27.293846 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8350e86-a9e3-458b-a9ef-51bc3c4e9b14 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:27.295472 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8350e86-a9e3-458b-a9ef-51bc3c4e9b14 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:27.306530 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 512/1015] 10.222.0.58 () {56 vars in 1112 bytes} [Tue Jul 26 13:34:27 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:34:27.307691 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96abe07-f655-48cb-8205-86da6d5aabe5 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:27.420957 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96abe07-f655-48cb-8205-86da6d5aabe5 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:27.457263 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a526c9-f467-4cfe-bb35-9f5902759302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:27.457920 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a526c9-f467-4cfe-bb35-9f5902759302 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:27.458202 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a526c9-f467-4cfe-bb35-9f5902759302 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:27.458474 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a526c9-f467-4cfe-bb35-9f5902759302 None admin] PATH_INFO: `/v3/services` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:27.459151 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a526c9-f467-4cfe-bb35-9f5902759302 None admin] RBAC: Adding query filter params (type=compute) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:27.459480 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a526c9-f467-4cfe-bb35-9f5902759302 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:27.461458 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a526c9-f467-4cfe-bb35-9f5902759302 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:27.466665 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 504/1016] 10.222.0.58 () {56 vars in 1112 bytes} [Tue Jul 26 13:34:27 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:27.472540 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8350e86-a9e3-458b-a9ef-51bc3c4e9b14 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:27.577861 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8350e86-a9e3-458b-a9ef-51bc3c4e9b14 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:27.594578 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:27.613548 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a0e987-55fa-4c33-8e09-394385c10c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:27.614183 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a0e987-55fa-4c33-8e09-394385c10c7d None admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:27.614469 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a0e987-55fa-4c33-8e09-394385c10c7d None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:27.614748 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a0e987-55fa-4c33-8e09-394385c10c7d None admin] PATH_INFO: `/v3/endpoints` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:27.615472 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a0e987-55fa-4c33-8e09-394385c10c7d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:27.616871 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a0e987-55fa-4c33-8e09-394385c10c7d None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:27.637558 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 513/1017] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 13:34:27 2022] POST /identity/v3/endpoints => generated 334 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:34:28.163994 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e76cbd-1f42-43ff-b460-4c0f591d97de None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:28.164537 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e76cbd-1f42-43ff-b460-4c0f591d97de None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:28.164537 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e76cbd-1f42-43ff-b460-4c0f591d97de None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:28.166113 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 505/1018] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:34:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:34:28.171054 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91587bd1-900f-4274-8aa1-5459194f907a None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:28.171710 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91587bd1-900f-4274-8aa1-5459194f907a None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:28.171920 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91587bd1-900f-4274-8aa1-5459194f907a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:28.190234 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-91587bd1-900f-4274-8aa1-5459194f907a None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:34:28.191426 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-91587bd1-900f-4274-8aa1-5459194f907a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:34:28.268356 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 514/1019] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:34:28 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 13:34:28.290639 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e76cbd-1f42-43ff-b460-4c0f591d97de None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:28.351991 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:28.397866 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e76cbd-1f42-43ff-b460-4c0f591d97de None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:28.435463 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60513046-3a64-46bb-8ac7-29b1b32f9e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:28.436142 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60513046-3a64-46bb-8ac7-29b1b32f9e93 None admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:28.436431 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60513046-3a64-46bb-8ac7-29b1b32f9e93 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:28.436706 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60513046-3a64-46bb-8ac7-29b1b32f9e93 None admin] PATH_INFO: `/v3/services` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:28.437397 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60513046-3a64-46bb-8ac7-29b1b32f9e93 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:28.438797 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60513046-3a64-46bb-8ac7-29b1b32f9e93 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:28.455743 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 506/1020] 10.222.0.58 () {60 vars in 1141 bytes} [Tue Jul 26 13:34:28 2022] POST /identity/v3/services => generated 242 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:34:29.310584 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fc8c79-7945-4e8a-90d2-4f01fe8f2fa6 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:29.311083 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fc8c79-7945-4e8a-90d2-4f01fe8f2fa6 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:29.311083 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fc8c79-7945-4e8a-90d2-4f01fe8f2fa6 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:29.312453 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 515/1021] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:34:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:34:29.317548 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dfd668-9ed5-4c6d-a779-bd5807f57865 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:29.317936 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dfd668-9ed5-4c6d-a779-bd5807f57865 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:29.318125 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dfd668-9ed5-4c6d-a779-bd5807f57865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:29.336544 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-03dfd668-9ed5-4c6d-a779-bd5807f57865 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:34:29.337760 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-03dfd668-9ed5-4c6d-a779-bd5807f57865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:34:29.410862 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 507/1022] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:34:29 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 13:34:29.435301 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fc8c79-7945-4e8a-90d2-4f01fe8f2fa6 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:29.536339 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:29.543608 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fc8c79-7945-4e8a-90d2-4f01fe8f2fa6 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:29.579733 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c1add4-1377-4df0-9b81-c732e4afc803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:29.580407 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c1add4-1377-4df0-9b81-c732e4afc803 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:29.580691 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c1add4-1377-4df0-9b81-c732e4afc803 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:29.580965 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c1add4-1377-4df0-9b81-c732e4afc803 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:29.582178 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c1add4-1377-4df0-9b81-c732e4afc803 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:29.583618 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c1add4-1377-4df0-9b81-c732e4afc803 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:29.590384 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 516/1023] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:34:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:29.591664 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dfd668-9ed5-4c6d-a779-bd5807f57865 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:29.639630 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:29.695435 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dfd668-9ed5-4c6d-a779-bd5807f57865 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:29.730359 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac09a8e-26a6-4c34-9a76-00f235be4009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:29.730984 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac09a8e-26a6-4c34-9a76-00f235be4009 None admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:29.731263 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac09a8e-26a6-4c34-9a76-00f235be4009 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:29.731535 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac09a8e-26a6-4c34-9a76-00f235be4009 None admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:29.732212 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac09a8e-26a6-4c34-9a76-00f235be4009 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:29.733654 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac09a8e-26a6-4c34-9a76-00f235be4009 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:29.755701 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 508/1024] 10.222.0.58 () {60 vars in 1132 bytes} [Tue Jul 26 13:34:29 2022] POST /identity/v3/users => generated 285 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 13:34:29.761989 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48b19ce-dc54-4269-b4b4-135dc548d00d None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:29.762273 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48b19ce-dc54-4269-b4b4-135dc548d00d None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:29.762574 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48b19ce-dc54-4269-b4b4-135dc548d00d None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:29.763991 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 517/1025] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:34:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:34:29.769829 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3541ca05-9ab4-4f17-8b2e-97472bae8b41 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:29.770133 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3541ca05-9ab4-4f17-8b2e-97472bae8b41 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:29.770415 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3541ca05-9ab4-4f17-8b2e-97472bae8b41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:29.790801 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-3541ca05-9ab4-4f17-8b2e-97472bae8b41 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:34:29.792018 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-3541ca05-9ab4-4f17-8b2e-97472bae8b41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:34:29.866553 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 509/1026] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:34:29 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 13:34:29.890773 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48b19ce-dc54-4269-b4b4-135dc548d00d None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:29.997590 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48b19ce-dc54-4269-b4b4-135dc548d00d None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:30.033312 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3b3196-1683-4aa3-a22a-172f38104375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:30.034008 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3b3196-1683-4aa3-a22a-172f38104375 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:30.034290 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3b3196-1683-4aa3-a22a-172f38104375 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:30.034624 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3b3196-1683-4aa3-a22a-172f38104375 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:30.051219 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3b3196-1683-4aa3-a22a-172f38104375 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:34:30.056778 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-1e3b3196-1683-4aa3-a22a-172f38104375 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 13:34:30.056778 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:30.056778 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:34:30.056778 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:34:30.056778 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:34:30.056778 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:34:30.056778 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:34:30.056778 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:34:30.056778 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:34:30.056778 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:34:30.056778 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:34:30.056778 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:34:30.056778 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:34:30.056778 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:34:30.056778 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:34:30.056778 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:34:30.057759 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:34:30.057759 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:34:30.057759 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:34:30.057759 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:34:30.057759 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:34:30.057759 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:34:30.057759 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:34:30.057759 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:34:30.057759 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:34:30.057759 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:34:30.057759 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:34:30.057759 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:34:30.057759 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:34:30.057759 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:34:30.057759 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:34:30.057759 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:34:30.058624 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:34:30.058624 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:34:30.058624 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:34:30.058624 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:34:30.058624 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:34:30.058624 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:34:30.058624 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:34:30.058624 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:34:30.058624 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:34:30.058624 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:34:30.058624 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 13:34:30.058624 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:34:30.058624 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 518/1027] 10.222.0.58 () {56 vars in 1120 bytes} [Tue Jul 26 13:34:29 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:34:30.063989 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3541ca05-9ab4-4f17-8b2e-97472bae8b41 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:30.096411 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:30.168777 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3541ca05-9ab4-4f17-8b2e-97472bae8b41 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:30.204063 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6be8ad-a692-41e0-9544-f6f732fa6b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:30.204686 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6be8ad-a692-41e0-9544-f6f732fa6b6e None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:30.204967 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6be8ad-a692-41e0-9544-f6f732fa6b6e None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:30.205241 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6be8ad-a692-41e0-9544-f6f732fa6b6e None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:30.205980 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6be8ad-a692-41e0-9544-f6f732fa6b6e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:30.206286 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6be8ad-a692-41e0-9544-f6f732fa6b6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:30.207644 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6be8ad-a692-41e0-9544-f6f732fa6b6e None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:30.220186 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 510/1028] 10.222.0.58 () {56 vars in 1115 bytes} [Tue Jul 26 13:34:30 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:34:30.221381 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3b3196-1683-4aa3-a22a-172f38104375 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:30.249846 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:30.325515 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3b3196-1683-4aa3-a22a-172f38104375 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:30.361167 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ebf315-99e0-491a-b59a-53b3864267f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:30.361885 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ebf315-99e0-491a-b59a-53b3864267f1 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:30.362170 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ebf315-99e0-491a-b59a-53b3864267f1 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:30.362457 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ebf315-99e0-491a-b59a-53b3864267f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:30.374580 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ebf315-99e0-491a-b59a-53b3864267f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:30.376026 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ebf315-99e0-491a-b59a-53b3864267f1 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:30.391334 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-17ebf315-99e0-491a-b59a-53b3864267f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:34:30.391334 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:30.391334 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:34:30.391334 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:34:30.391334 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:34:30.391334 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:34:30.391334 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:34:30.391334 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:34:30.391334 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:34:30.391334 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:34:30.391334 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:34:30.391334 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:34:30.391334 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 13:34:30.391334 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 13:34:30.391334 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 13:34:30.392263 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 13:34:30.392263 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:34:30.392263 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:34:30.392263 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:34:30.392263 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:34:30.392263 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:34:30.392263 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:34:30.392263 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:34:30.392263 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:34:30.392263 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:34:30.392263 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:34:30.392263 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:34:30.392263 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:34:30.392263 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:34:30.392263 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:34:30.392263 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:34:30.392263 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:34:30.393156 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 13:34:30.393156 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 13:34:30.393156 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:34:30.393156 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:34:30.393156 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 519/1029] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:34:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 13:34:30.398203 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6be8ad-a692-41e0-9544-f6f732fa6b6e None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:30.488342 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7ca537-4e90-4ad6-96f0-c4d76d53c52e None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:30.488556 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7ca537-4e90-4ad6-96f0-c4d76d53c52e None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:30.488836 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7ca537-4e90-4ad6-96f0-c4d76d53c52e None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:30.490383 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 520/1030] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:34:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:34:30.495697 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643b0d8d-f05f-43fb-80b1-33d2dbd9b1c3 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:30.496220 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643b0d8d-f05f-43fb-80b1-33d2dbd9b1c3 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:30.496502 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643b0d8d-f05f-43fb-80b1-33d2dbd9b1c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:30.497033 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:30.505124 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6be8ad-a692-41e0-9544-f6f732fa6b6e None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:30.514735 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-643b0d8d-f05f-43fb-80b1-33d2dbd9b1c3 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:34:30.515934 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-643b0d8d-f05f-43fb-80b1-33d2dbd9b1c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:34:30.540872 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b0a7c3-dc2a-42db-b88a-296142c0f196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:30.541519 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b0a7c3-dc2a-42db-b88a-296142c0f196 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:30.541812 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b0a7c3-dc2a-42db-b88a-296142c0f196 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:30.542090 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b0a7c3-dc2a-42db-b88a-296142c0f196 None admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:30.542784 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b0a7c3-dc2a-42db-b88a-296142c0f196 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:30.543093 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b0a7c3-dc2a-42db-b88a-296142c0f196 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:30.544473 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b0a7c3-dc2a-42db-b88a-296142c0f196 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:30.558136 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 511/1031] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Jul 26 13:34:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:30.563670 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b0a7c3-dc2a-42db-b88a-296142c0f196 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:30.591313 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 521/1032] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:34:30 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 13:34:30.614377 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643b0d8d-f05f-43fb-80b1-33d2dbd9b1c3 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:30.626340 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:30.670987 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b0a7c3-dc2a-42db-b88a-296142c0f196 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:30.711195 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ebe95d-d900-4436-9345-4af511523b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:30.712102 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ebe95d-d900-4436-9345-4af511523b2e None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:30.712390 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ebe95d-d900-4436-9345-4af511523b2e None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:30.712668 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ebe95d-d900-4436-9345-4af511523b2e None admin] PATH_INFO: `/v3/users/nova` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:30.720373 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643b0d8d-f05f-43fb-80b1-33d2dbd9b1c3 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:30.720622 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ebe95d-d900-4436-9345-4af511523b2e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:30.722053 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ebe95d-d900-4436-9345-4af511523b2e None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:30.735854 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-e0ebe95d-d900-4436-9345-4af511523b2e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 26 13:34:30.735854 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:30.735854 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:34:30.735854 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:34:30.735854 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:34:30.735854 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:34:30.735854 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:34:30.735854 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:34:30.735854 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:34:30.735854 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:34:30.735854 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:34:30.735854 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:34:30.735854 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 13:34:30.735854 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 13:34:30.735854 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 13:34:30.735854 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 13:34:30.736852 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:34:30.736852 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:34:30.736852 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 13:34:30.736852 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 13:34:30.736852 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 13:34:30.736852 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 13:34:30.736852 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:34:30.736852 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:34:30.736852 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:34:30.736852 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:34:30.736852 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:34:30.736852 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:34:30.736852 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:34:30.736852 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:34:30.736852 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:34:30.736852 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:34:30.736852 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:34:30.736852 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:34:30.738188 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:34:30.738188 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:34:30.738188 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 13:34:30.738188 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 13:34:30.738188 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 13:34:30.738188 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 13:34:30.738188 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 13:34:30.738188 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 13:34:30.738188 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 26 13:34:30.738188 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:34:30.738188 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 512/1033] 10.222.0.58 () {56 vars in 1093 bytes} [Tue Jul 26 13:34:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 13:34:30.742708 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ebe95d-d900-4436-9345-4af511523b2e None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:30.759603 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897fdbd9-9dd0-4389-bd42-07636d01b21a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:30.759797 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:30.760256 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897fdbd9-9dd0-4389-bd42-07636d01b21a None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:30.760544 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897fdbd9-9dd0-4389-bd42-07636d01b21a None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:30.760823 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897fdbd9-9dd0-4389-bd42-07636d01b21a None admin] PATH_INFO: `/v3/services/image` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:30.761459 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897fdbd9-9dd0-4389-bd42-07636d01b21a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:30.762846 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897fdbd9-9dd0-4389-bd42-07636d01b21a None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:30.772362 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-897fdbd9-9dd0-4389-bd42-07636d01b21a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 26 13:34:30.772362 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:30.772362 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 13:34:30.772362 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 13:34:30.772362 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 13:34:30.772362 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 13:34:30.772362 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 13:34:30.772362 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 13:34:30.772362 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 26 13:34:30.772362 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:34:30.772362 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 13:34:30.772362 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:34:30.772362 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:30.772362 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:34:30.772362 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:34:30.772362 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:34:30.772362 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:34:30.773385 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:34:30.773385 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:34:30.773385 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:34:30.773385 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:34:30.773385 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:34:30.773385 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:34:30.773385 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 13:34:30.773385 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 13:34:30.773385 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 13:34:30.773385 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 13:34:30.773385 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:34:30.773385 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:34:30.773385 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:34:30.773385 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:34:30.773385 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:34:30.773385 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:34:30.774697 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:34:30.774697 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:34:30.774697 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:34:30.774697 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:34:30.774697 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:34:30.774697 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:34:30.774697 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:34:30.774697 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:34:30.774697 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:34:30.774697 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:34:30.774697 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 13:34:30.774697 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 13:34:30.774697 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 26 13:34:30.774697 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:34:30.778500 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 522/1034] 10.222.0.58 () {56 vars in 1105 bytes} [Tue Jul 26 13:34:30 2022] GET /identity/v3/services/image => generated 86 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 13:34:30.779598 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897fdbd9-9dd0-4389-bd42-07636d01b21a None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:30.853982 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ebe95d-d900-4436-9345-4af511523b2e None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:30.857914 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:30.889206 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b46f1c-075a-433c-a8ff-eb4ff99e69f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:30.890207 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b46f1c-075a-433c-a8ff-eb4ff99e69f6 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:30.890631 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b46f1c-075a-433c-a8ff-eb4ff99e69f6 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:30.890831 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b46f1c-075a-433c-a8ff-eb4ff99e69f6 None admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:30.891544 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b46f1c-075a-433c-a8ff-eb4ff99e69f6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:30.891854 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b46f1c-075a-433c-a8ff-eb4ff99e69f6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:30.892027 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897fdbd9-9dd0-4389-bd42-07636d01b21a None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:30.893273 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b46f1c-075a-433c-a8ff-eb4ff99e69f6 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:30.914814 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 513/1035] 10.222.0.58 () {56 vars in 1133 bytes} [Tue Jul 26 13:34:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:30.920676 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b46f1c-075a-433c-a8ff-eb4ff99e69f6 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:30.933818 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170029ad-6df9-42df-91de-440f87d39aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:30.934537 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170029ad-6df9-42df-91de-440f87d39aef None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:30.934735 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170029ad-6df9-42df-91de-440f87d39aef None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:30.935015 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170029ad-6df9-42df-91de-440f87d39aef None admin] PATH_INFO: `/v3/services` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:30.935709 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170029ad-6df9-42df-91de-440f87d39aef None admin] RBAC: Adding query filter params (name=image) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:30.936028 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170029ad-6df9-42df-91de-440f87d39aef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:30.937389 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170029ad-6df9-42df-91de-440f87d39aef None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:30.947744 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 523/1036] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:34:30 2022] GET /identity/v3/services?name=image => generated 122 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:30.949047 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170029ad-6df9-42df-91de-440f87d39aef None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:31.004245 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:31.044384 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b46f1c-075a-433c-a8ff-eb4ff99e69f6 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:31.063079 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170029ad-6df9-42df-91de-440f87d39aef None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:31.082231 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831937aa-3a2c-450d-a8bb-691bb405f292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:31.082891 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831937aa-3a2c-450d-a8bb-691bb405f292 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:31.083255 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831937aa-3a2c-450d-a8bb-691bb405f292 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:31.083530 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831937aa-3a2c-450d-a8bb-691bb405f292 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:31.097312 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5193f1af-ac69-47e3-ba85-572bc2e69ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:31.098017 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5193f1af-ac69-47e3-ba85-572bc2e69ce8 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:31.098302 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5193f1af-ac69-47e3-ba85-572bc2e69ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:31.098581 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5193f1af-ac69-47e3-ba85-572bc2e69ce8 None admin] PATH_INFO: `/v3/services` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:31.099256 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5193f1af-ac69-47e3-ba85-572bc2e69ce8 None admin] RBAC: Adding query filter params (type=image) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:31.099560 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5193f1af-ac69-47e3-ba85-572bc2e69ce8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:31.100957 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5193f1af-ac69-47e3-ba85-572bc2e69ce8 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:31.103057 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:31.111177 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 524/1037] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:34:30 2022] GET /identity/v3/services?type=image => generated 350 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:34:31.112196 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5193f1af-ac69-47e3-ba85-572bc2e69ce8 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:31.164590 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831937aa-3a2c-450d-a8bb-691bb405f292 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:31.166130 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831937aa-3a2c-450d-a8bb-691bb405f292 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:31.220107 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5193f1af-ac69-47e3-ba85-572bc2e69ce8 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:31.244385 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:31.255656 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77d312d-8dcb-4050-8516-95d8071fe5d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:31.256997 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77d312d-8dcb-4050-8516-95d8071fe5d6 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:31.257423 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77d312d-8dcb-4050-8516-95d8071fe5d6 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:31.257863 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77d312d-8dcb-4050-8516-95d8071fe5d6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:31.258749 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77d312d-8dcb-4050-8516-95d8071fe5d6 None admin] RBAC: Adding query filter params (interface=public, service_id=0180544c912b42a2a4aa6c040e8f57d7, region_id=RegionOne) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:31.259248 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77d312d-8dcb-4050-8516-95d8071fe5d6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:31.260610 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77d312d-8dcb-4050-8516-95d8071fe5d6 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:31.270293 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 525/1038] 10.222.0.58 () {56 vars in 1251 bytes} [Tue Jul 26 13:34:31 2022] GET /identity/v3/endpoints?service_id=0180544c912b42a2a4aa6c040e8f57d7&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:31.271259 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77d312d-8dcb-4050-8516-95d8071fe5d6 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:31.287954 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 514/1039] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Jul 26 13:34:30 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 13:34:31.289076 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831937aa-3a2c-450d-a8bb-691bb405f292 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:31.342412 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:31.384712 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77d312d-8dcb-4050-8516-95d8071fe5d6 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:31.393221 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831937aa-3a2c-450d-a8bb-691bb405f292 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:31.420923 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fa39ee-89a1-445b-8be3-4a5bdae5dd78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:31.421820 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fa39ee-89a1-445b-8be3-4a5bdae5dd78 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:31.422103 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fa39ee-89a1-445b-8be3-4a5bdae5dd78 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:31.422375 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fa39ee-89a1-445b-8be3-4a5bdae5dd78 None admin] PATH_INFO: `/v3/services` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:31.423122 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fa39ee-89a1-445b-8be3-4a5bdae5dd78 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:31.424600 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fa39ee-89a1-445b-8be3-4a5bdae5dd78 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:31.429350 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71a9d92-9fb8-4f2c-aeeb-9dac7e72a478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:31.430037 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71a9d92-9fb8-4f2c-aeeb-9dac7e72a478 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:31.430325 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71a9d92-9fb8-4f2c-aeeb-9dac7e72a478 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:31.430602 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71a9d92-9fb8-4f2c-aeeb-9dac7e72a478 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:31.441583 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 526/1040] 10.222.0.58 () {56 vars in 1087 bytes} [Tue Jul 26 13:34:31 2022] GET /identity/v3/services => generated 1957 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:34:31.452641 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71a9d92-9fb8-4f2c-aeeb-9dac7e72a478 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:31.454277 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71a9d92-9fb8-4f2c-aeeb-9dac7e72a478 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:31.471703 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-c71a9d92-9fb8-4f2c-aeeb-9dac7e72a478 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:34:31.471703 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:31.471703 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:34:31.471703 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:34:31.471703 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:34:31.471703 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:34:31.471703 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:34:31.471703 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:34:31.471703 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:34:31.471703 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:34:31.471703 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:34:31.471703 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:34:31.471703 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 13:34:31.471703 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 13:34:31.471703 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 13:34:31.472684 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 13:34:31.472684 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:34:31.472684 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:34:31.472684 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:34:31.472684 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:34:31.472684 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:34:31.472684 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:34:31.472684 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:34:31.472684 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:34:31.472684 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:34:31.472684 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:34:31.472684 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:34:31.472684 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:34:31.472684 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:34:31.472684 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:34:31.472684 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:34:31.472684 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:34:31.473655 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 13:34:31.473655 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 13:34:31.473655 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:34:31.473655 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:34:31.473655 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 515/1041] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:34:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:34:31.478699 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fa39ee-89a1-445b-8be3-4a5bdae5dd78 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:31.588408 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fa39ee-89a1-445b-8be3-4a5bdae5dd78 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:31.617206 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:31.625393 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64fd48c-2f6c-42ce-bf49-90f402637557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:31.626067 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64fd48c-2f6c-42ce-bf49-90f402637557 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:31.626349 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64fd48c-2f6c-42ce-bf49-90f402637557 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:31.626626 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64fd48c-2f6c-42ce-bf49-90f402637557 None admin] PATH_INFO: `/v3/domains` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:31.627327 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64fd48c-2f6c-42ce-bf49-90f402637557 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:31.627632 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64fd48c-2f6c-42ce-bf49-90f402637557 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:31.628994 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64fd48c-2f6c-42ce-bf49-90f402637557 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:31.642181 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 527/1042] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Jul 26 13:34:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:34:31.647922 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71a9d92-9fb8-4f2c-aeeb-9dac7e72a478 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:31.674745 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:31.752583 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71a9d92-9fb8-4f2c-aeeb-9dac7e72a478 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:31.786972 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951338a0-b1fb-45af-acfe-7638681bd222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:31.788141 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951338a0-b1fb-45af-acfe-7638681bd222 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:31.788425 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951338a0-b1fb-45af-acfe-7638681bd222 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:31.788701 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951338a0-b1fb-45af-acfe-7638681bd222 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:31.795580 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951338a0-b1fb-45af-acfe-7638681bd222 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:31.797251 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951338a0-b1fb-45af-acfe-7638681bd222 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:31.809410 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-951338a0-b1fb-45af-acfe-7638681bd222 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 13:34:31.809410 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:31.809410 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:34:31.809410 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:34:31.809410 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:34:31.809410 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:34:31.809410 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:34:31.809410 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:34:31.809410 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:34:31.809410 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:34:31.809410 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:34:31.809410 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:34:31.809410 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 13:34:31.809410 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 13:34:31.809410 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 13:34:31.810324 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 13:34:31.810324 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:34:31.810324 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:34:31.810324 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:34:31.810324 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:34:31.810324 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:34:31.810324 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:34:31.810324 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:34:31.810324 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:34:31.810324 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:34:31.810324 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:34:31.810324 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:34:31.810324 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:34:31.810324 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:34:31.810324 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:34:31.810324 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:34:31.810324 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:34:31.811213 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 13:34:31.811213 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 13:34:31.811213 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 13:34:31.811213 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 13:34:31.811213 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 13:34:31.811213 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 13:34:31.811213 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 13:34:31.811213 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:34:31.811213 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 516/1043] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Jul 26 13:34:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:34:31.817908 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64fd48c-2f6c-42ce-bf49-90f402637557 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:31.858319 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7a023a-ecfe-49bb-9367-abcf738f96b7 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:31.858601 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7a023a-ecfe-49bb-9367-abcf738f96b7 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:31.859180 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7a023a-ecfe-49bb-9367-abcf738f96b7 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:31.860401 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 517/1044] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:34:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 13:34:31.865171 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfc09e2-3d40-4457-ae71-63b0feb422f4 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:31.865531 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfc09e2-3d40-4457-ae71-63b0feb422f4 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:31.866312 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfc09e2-3d40-4457-ae71-63b0feb422f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:31.885136 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-0cfc09e2-3d40-4457-ae71-63b0feb422f4 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:34:31.886358 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-0cfc09e2-3d40-4457-ae71-63b0feb422f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:34:31.933721 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64fd48c-2f6c-42ce-bf49-90f402637557 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:31.961781 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 518/1045] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:34:31 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 13:34:31.973507 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc55a53-9976-4368-98b1-b4f6927355a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:31.974207 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc55a53-9976-4368-98b1-b4f6927355a3 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:31.974487 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc55a53-9976-4368-98b1-b4f6927355a3 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:31.974763 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc55a53-9976-4368-98b1-b4f6927355a3 None admin] PATH_INFO: `/v3/projects` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:31.976089 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc55a53-9976-4368-98b1-b4f6927355a3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:31.976403 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc55a53-9976-4368-98b1-b4f6927355a3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:31.977914 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc55a53-9976-4368-98b1-b4f6927355a3 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:31.985099 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfc09e2-3d40-4457-ae71-63b0feb422f4 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:31.991840 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 528/1046] 10.222.0.58 () {56 vars in 1148 bytes} [Tue Jul 26 13:34:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:31.998031 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc55a53-9976-4368-98b1-b4f6927355a3 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:32.001383 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:32.060304 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:32.100437 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfc09e2-3d40-4457-ae71-63b0feb422f4 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:32.114530 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc55a53-9976-4368-98b1-b4f6927355a3 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:32.136850 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813c1714-f825-43f2-8837-cb236b8a1e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:32.137525 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813c1714-f825-43f2-8837-cb236b8a1e66 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:32.137917 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813c1714-f825-43f2-8837-cb236b8a1e66 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:32.138189 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813c1714-f825-43f2-8837-cb236b8a1e66 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:32.149617 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813c1714-f825-43f2-8837-cb236b8a1e66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:34:32.150821 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dde84a5-9f6f-4319-a1aa-71b44108db3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:32.151442 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dde84a5-9f6f-4319-a1aa-71b44108db3e None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:32.151724 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dde84a5-9f6f-4319-a1aa-71b44108db3e None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:32.151999 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dde84a5-9f6f-4319-a1aa-71b44108db3e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:32.152791 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dde84a5-9f6f-4319-a1aa-71b44108db3e None admin] RBAC: Adding query filter params (role.id=ac11ff5f40194d7d98f81a5429aa3826, scope.project.id=1e8bb4385c204210a1bc77b1f0d654f3, user.id=c69161cd520b4b29bf50f3bf2f1c033b) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:32.153174 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dde84a5-9f6f-4319-a1aa-71b44108db3e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:32.154401 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-813c1714-f825-43f2-8837-cb236b8a1e66 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 13:34:32.154401 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:32.154401 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:34:32.154401 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:34:32.154401 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:34:32.154401 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:34:32.154401 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:34:32.154401 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:34:32.154401 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:34:32.154401 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:34:32.154401 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:34:32.154401 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:34:32.154401 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:34:32.154401 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:34:32.154401 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:34:32.154401 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:34:32.155307 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:34:32.155307 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:34:32.155307 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:34:32.155307 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:34:32.155307 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:34:32.155307 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:34:32.155307 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:34:32.155307 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:34:32.155307 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:34:32.155307 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:34:32.155307 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:34:32.155307 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:34:32.155307 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:34:32.155307 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:34:32.155307 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:34:32.155307 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:34:32.156151 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:34:32.156151 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:34:32.156151 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:34:32.156151 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:34:32.156151 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:34:32.156151 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:34:32.156151 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:34:32.156151 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:34:32.156151 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:34:32.156151 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:34:32.156151 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 13:34:32.156151 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:34:32.156802 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dde84a5-9f6f-4319-a1aa-71b44108db3e None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:32.156873 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 519/1047] 10.222.0.58 () {56 vars in 1096 bytes} [Tue Jul 26 13:34:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 13:34:32.161348 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813c1714-f825-43f2-8837-cb236b8a1e66 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:32.166603 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 529/1048] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Jul 26 13:34:31 2022] GET /identity/v3/role_assignments?user.id=c69161cd520b4b29bf50f3bf2f1c033b&scope.project.id=1e8bb4385c204210a1bc77b1f0d654f3&role.id=ac11ff5f40194d7d98f81a5429aa3826 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:32.258333 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:32.287130 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813c1714-f825-43f2-8837-cb236b8a1e66 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:32.324745 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed42c9d4-31ab-4cc2-bfdb-aa33e06ccd60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:32.325487 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed42c9d4-31ab-4cc2-bfdb-aa33e06ccd60 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:32.325681 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed42c9d4-31ab-4cc2-bfdb-aa33e06ccd60 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:32.325956 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed42c9d4-31ab-4cc2-bfdb-aa33e06ccd60 None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:32.326680 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed42c9d4-31ab-4cc2-bfdb-aa33e06ccd60 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:32.326987 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed42c9d4-31ab-4cc2-bfdb-aa33e06ccd60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:32.328351 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed42c9d4-31ab-4cc2-bfdb-aa33e06ccd60 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:32.337409 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 520/1049] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Jul 26 13:34:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:34:32.343351 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dde84a5-9f6f-4319-a1aa-71b44108db3e None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:32.446228 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dde84a5-9f6f-4319-a1aa-71b44108db3e None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:32.459045 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:32.482092 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e2a088-1b1c-41ae-a9b4-f425faf25079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:32.482739 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e2a088-1b1c-41ae-a9b4-f425faf25079 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:32.483021 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e2a088-1b1c-41ae-a9b4-f425faf25079 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:32.483295 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e2a088-1b1c-41ae-a9b4-f425faf25079 None admin] PATH_INFO: `/v3/users/95c99d354cee4bcbb241fa56ef59110c` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:32.497386 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e2a088-1b1c-41ae-a9b4-f425faf25079 None admin] RBAC: Authorizing `identity:get_user(user_id=95c99d354cee4bcbb241fa56ef59110c)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:32.498907 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e2a088-1b1c-41ae-a9b4-f425faf25079 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:32.505209 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 530/1050] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Jul 26 13:34:32 2022] GET /identity/v3/users/95c99d354cee4bcbb241fa56ef59110c => generated 285 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:32.506594 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed42c9d4-31ab-4cc2-bfdb-aa33e06ccd60 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:32.612187 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed42c9d4-31ab-4cc2-bfdb-aa33e06ccd60 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:32.647952 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8001c0ba-3135-48d9-9163-fa3ce01fc2c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:32.648575 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8001c0ba-3135-48d9-9163-fa3ce01fc2c4 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:32.648859 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8001c0ba-3135-48d9-9163-fa3ce01fc2c4 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:32.649132 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8001c0ba-3135-48d9-9163-fa3ce01fc2c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:32.652783 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:32.719193 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8001c0ba-3135-48d9-9163-fa3ce01fc2c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:32.720547 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8001c0ba-3135-48d9-9163-fa3ce01fc2c4 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:32.824166 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 521/1051] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Jul 26 13:34:32 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 13:34:32.826550 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e2a088-1b1c-41ae-a9b4-f425faf25079 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:32.928578 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e2a088-1b1c-41ae-a9b4-f425faf25079 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:32.962091 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f297600d-b62f-4271-b5be-8a2d17c2f76a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:32.962743 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f297600d-b62f-4271-b5be-8a2d17c2f76a None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:32.963019 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f297600d-b62f-4271-b5be-8a2d17c2f76a None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:32.963289 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f297600d-b62f-4271-b5be-8a2d17c2f76a None admin] PATH_INFO: `/v3/projects/44d0149bb4584137bf865ded6722f055` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:32.964442 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f297600d-b62f-4271-b5be-8a2d17c2f76a None admin] RBAC: Authorizing `identity:get_project(project_id=44d0149bb4584137bf865ded6722f055)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:32.965901 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f297600d-b62f-4271-b5be-8a2d17c2f76a None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:32.972625 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 531/1052] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Jul 26 13:34:32 2022] GET /identity/v3/projects/44d0149bb4584137bf865ded6722f055 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:32.974178 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8001c0ba-3135-48d9-9163-fa3ce01fc2c4 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:33.011017 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:33.083866 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8001c0ba-3135-48d9-9163-fa3ce01fc2c4 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:33.119144 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f26f46e-51df-45cc-882e-24be53a8ac69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:33.119795 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f26f46e-51df-45cc-882e-24be53a8ac69 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:33.120084 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f26f46e-51df-45cc-882e-24be53a8ac69 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:33.120359 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f26f46e-51df-45cc-882e-24be53a8ac69 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:33.121147 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f26f46e-51df-45cc-882e-24be53a8ac69 None admin] RBAC: Adding query filter params (role.id=6e0201a9a6484f7d8b112b1eafff70bd, scope.project.id=44d0149bb4584137bf865ded6722f055, user.id=95c99d354cee4bcbb241fa56ef59110c) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:33.121462 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f26f46e-51df-45cc-882e-24be53a8ac69 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:33.122920 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f26f46e-51df-45cc-882e-24be53a8ac69 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:33.134691 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 522/1053] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Jul 26 13:34:32 2022] GET /identity/v3/role_assignments?user.id=95c99d354cee4bcbb241fa56ef59110c&scope.project.id=44d0149bb4584137bf865ded6722f055&role.id=6e0201a9a6484f7d8b112b1eafff70bd => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:33.460180 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d033fe8e-660d-4980-9645-adf3f7376da2 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:33.460783 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d033fe8e-660d-4980-9645-adf3f7376da2 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:33.460783 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d033fe8e-660d-4980-9645-adf3f7376da2 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:33.462280 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 532/1054] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:34:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:34:33.467387 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be955485-7779-498c-9e0e-100bf09583b3 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:33.467678 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be955485-7779-498c-9e0e-100bf09583b3 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:33.467955 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be955485-7779-498c-9e0e-100bf09583b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:33.487103 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-be955485-7779-498c-9e0e-100bf09583b3 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:34:33.488277 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-be955485-7779-498c-9e0e-100bf09583b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:34:33.562174 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 523/1055] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:34:33 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 13:34:33.579646 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d033fe8e-660d-4980-9645-adf3f7376da2 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:33.621529 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:33.683755 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d033fe8e-660d-4980-9645-adf3f7376da2 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:33.718486 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f29d99-5875-4d17-9f70-89ee6cd32fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:33.719053 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f29d99-5875-4d17-9f70-89ee6cd32fd7 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:33.719331 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f29d99-5875-4d17-9f70-89ee6cd32fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:33.719604 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f29d99-5875-4d17-9f70-89ee6cd32fd7 None admin] PATH_INFO: `/v3/services/image` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:33.720227 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f29d99-5875-4d17-9f70-89ee6cd32fd7 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:33.721602 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f29d99-5875-4d17-9f70-89ee6cd32fd7 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:33.733128 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-e3f29d99-5875-4d17-9f70-89ee6cd32fd7 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 26 13:34:33.733128 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:33.733128 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 13:34:33.733128 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 13:34:33.733128 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 13:34:33.733128 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 13:34:33.733128 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 13:34:33.733128 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 13:34:33.733128 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 26 13:34:33.733128 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:34:33.733128 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 13:34:33.733128 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:34:33.733128 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:33.733128 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:34:33.733128 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:34:33.733128 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:34:33.733128 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:34:33.734192 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:34:33.734192 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:34:33.734192 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:34:33.734192 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:34:33.734192 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:34:33.734192 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:34:33.734192 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 13:34:33.734192 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 13:34:33.734192 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 13:34:33.734192 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 13:34:33.734192 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:34:33.734192 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:34:33.734192 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:34:33.734192 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:34:33.734192 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:34:33.734192 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:34:33.735035 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:34:33.735035 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:34:33.735035 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:34:33.735035 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:34:33.735035 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:34:33.735035 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:34:33.735035 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:34:33.735035 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:34:33.735035 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:34:33.735035 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:34:33.735035 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 13:34:33.735035 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 13:34:33.735035 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 26 13:34:33.735035 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:34:33.737825 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 533/1056] 10.222.0.58 () {56 vars in 1105 bytes} [Tue Jul 26 13:34:33 2022] GET /identity/v3/services/image => generated 86 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:34:33.740266 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be955485-7779-498c-9e0e-100bf09583b3 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:33.770810 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:33.844078 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be955485-7779-498c-9e0e-100bf09583b3 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:33.878560 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58034f8-6481-484b-adbf-ecd618d39f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:33.879183 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58034f8-6481-484b-adbf-ecd618d39f92 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:33.879469 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58034f8-6481-484b-adbf-ecd618d39f92 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:33.879743 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58034f8-6481-484b-adbf-ecd618d39f92 None admin] PATH_INFO: `/v3/services` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:33.880418 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58034f8-6481-484b-adbf-ecd618d39f92 None admin] RBAC: Adding query filter params (name=image) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:33.880723 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58034f8-6481-484b-adbf-ecd618d39f92 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:33.882139 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58034f8-6481-484b-adbf-ecd618d39f92 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:33.891752 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 524/1057] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:34:33 2022] GET /identity/v3/services?name=image => generated 122 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:33.892689 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f29d99-5875-4d17-9f70-89ee6cd32fd7 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:33.995642 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f29d99-5875-4d17-9f70-89ee6cd32fd7 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:33.997516 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:34.036956 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bcdeac-9079-468d-811b-07b07f5e60dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:34.038421 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bcdeac-9079-468d-811b-07b07f5e60dd None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:34.038847 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bcdeac-9079-468d-811b-07b07f5e60dd None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:34.039272 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bcdeac-9079-468d-811b-07b07f5e60dd None admin] PATH_INFO: `/v3/services` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:34.040105 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bcdeac-9079-468d-811b-07b07f5e60dd None admin] RBAC: Adding query filter params (type=image) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:34.040592 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bcdeac-9079-468d-811b-07b07f5e60dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:34.042010 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bcdeac-9079-468d-811b-07b07f5e60dd None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:34.052157 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 534/1058] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:34:33 2022] GET /identity/v3/services?type=image => generated 350 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:34:34.053265 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58034f8-6481-484b-adbf-ecd618d39f92 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:34.149081 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:34.156858 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58034f8-6481-484b-adbf-ecd618d39f92 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:34.167916 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc62d589-f4fb-4a8f-a41a-6429a3da9ff5 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:34.168210 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc62d589-f4fb-4a8f-a41a-6429a3da9ff5 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:34.168483 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc62d589-f4fb-4a8f-a41a-6429a3da9ff5 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:34.170141 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 535/1059] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:34:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:34:34.175067 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477a041d-2675-432e-b3bf-3b0cd99a4d89 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:34.175356 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477a041d-2675-432e-b3bf-3b0cd99a4d89 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:34.175633 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477a041d-2675-432e-b3bf-3b0cd99a4d89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:34.194844 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dca36e8-a05d-461e-b605-779f7f16671b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:34.195347 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-477a041d-2675-432e-b3bf-3b0cd99a4d89 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:34:34.195497 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dca36e8-a05d-461e-b605-779f7f16671b None admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:34.195735 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dca36e8-a05d-461e-b605-779f7f16671b None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:34.196012 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dca36e8-a05d-461e-b605-779f7f16671b None admin] PATH_INFO: `/v3/endpoints` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:34.196540 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-477a041d-2675-432e-b3bf-3b0cd99a4d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:34:34.196723 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dca36e8-a05d-461e-b605-779f7f16671b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:34.198157 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dca36e8-a05d-461e-b605-779f7f16671b None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:34.221858 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 525/1060] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 13:34:34 2022] POST /identity/v3/endpoints => generated 327 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:34:34.297706 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 536/1061] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:34:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 13:34:34.314958 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dca36e8-a05d-461e-b605-779f7f16671b None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:34.425541 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dca36e8-a05d-461e-b605-779f7f16671b None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:34.461664 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c060961c-96ea-4da5-96f4-680352b14a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:34.462317 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c060961c-96ea-4da5-96f4-680352b14a64 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:34.462598 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c060961c-96ea-4da5-96f4-680352b14a64 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:34.462872 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c060961c-96ea-4da5-96f4-680352b14a64 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:34.473229 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c060961c-96ea-4da5-96f4-680352b14a64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:34:34.478632 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-c060961c-96ea-4da5-96f4-680352b14a64 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 13:34:34.478632 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:34.478632 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:34:34.478632 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:34:34.478632 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:34:34.478632 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:34:34.478632 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:34:34.478632 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:34:34.478632 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:34:34.478632 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:34:34.478632 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:34:34.478632 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:34:34.478632 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:34:34.478632 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:34:34.478632 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:34:34.478632 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:34:34.479546 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:34:34.479546 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:34:34.479546 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:34:34.479546 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:34:34.479546 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:34:34.479546 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:34:34.479546 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:34:34.479546 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:34:34.479546 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:34:34.479546 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:34:34.479546 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:34:34.479546 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:34:34.479546 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:34:34.479546 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:34:34.479546 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:34:34.479546 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:34:34.480670 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:34:34.480670 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:34:34.480670 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:34:34.480670 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:34:34.480670 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:34:34.480670 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:34:34.480670 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:34:34.480670 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:34:34.480670 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:34:34.480670 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:34:34.480670 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 13:34:34.480670 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:34:34.480670 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 526/1062] 10.222.0.58 () {56 vars in 1120 bytes} [Tue Jul 26 13:34:34 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:34:34.485861 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477a041d-2675-432e-b3bf-3b0cd99a4d89 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:34.578349 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:34.588742 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477a041d-2675-432e-b3bf-3b0cd99a4d89 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:34.623383 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b51e850-689f-40b1-8f3e-df06684d3f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:34.624011 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b51e850-689f-40b1-8f3e-df06684d3f1c None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:34.624294 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b51e850-689f-40b1-8f3e-df06684d3f1c None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:34.624571 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b51e850-689f-40b1-8f3e-df06684d3f1c None admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:34.625288 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b51e850-689f-40b1-8f3e-df06684d3f1c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:34.625629 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b51e850-689f-40b1-8f3e-df06684d3f1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:34.626991 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b51e850-689f-40b1-8f3e-df06684d3f1c None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:34.640676 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 537/1063] 10.222.0.58 () {56 vars in 1115 bytes} [Tue Jul 26 13:34:34 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:34.642345 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c060961c-96ea-4da5-96f4-680352b14a64 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:34.655494 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:34.751288 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c060961c-96ea-4da5-96f4-680352b14a64 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:34.786346 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64680116-21f3-44d0-9dcc-ff4ecd22746d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:34.787048 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64680116-21f3-44d0-9dcc-ff4ecd22746d None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:34.787890 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64680116-21f3-44d0-9dcc-ff4ecd22746d None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:34.788171 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64680116-21f3-44d0-9dcc-ff4ecd22746d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:34.799524 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64680116-21f3-44d0-9dcc-ff4ecd22746d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:34.800971 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64680116-21f3-44d0-9dcc-ff4ecd22746d None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:34.816425 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-64680116-21f3-44d0-9dcc-ff4ecd22746d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:34:34.816425 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:34.816425 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:34:34.816425 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:34:34.816425 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:34:34.816425 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:34:34.816425 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:34:34.816425 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:34:34.816425 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:34:34.816425 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:34:34.816425 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:34:34.816425 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:34:34.816425 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 13:34:34.816425 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 13:34:34.816425 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 13:34:34.817346 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 13:34:34.817346 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:34:34.817346 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:34:34.817346 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:34:34.817346 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:34:34.817346 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:34:34.817346 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:34:34.817346 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:34:34.817346 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:34:34.817346 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:34:34.817346 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:34:34.817346 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:34:34.817346 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:34:34.817346 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:34:34.817346 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:34:34.817346 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:34:34.817346 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:34:34.818570 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 13:34:34.818570 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 13:34:34.818570 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:34:34.818570 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:34:34.821545 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 527/1064] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:34:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:34:34.823628 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b51e850-689f-40b1-8f3e-df06684d3f1c None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:34.925867 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b51e850-689f-40b1-8f3e-df06684d3f1c None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:34.959335 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1176ac-44a1-43cd-8888-fcba8cfa98f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:34.959960 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1176ac-44a1-43cd-8888-fcba8cfa98f5 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:34.960239 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1176ac-44a1-43cd-8888-fcba8cfa98f5 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:34.960511 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1176ac-44a1-43cd-8888-fcba8cfa98f5 None admin] PATH_INFO: `/v3/domains` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:34.961203 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1176ac-44a1-43cd-8888-fcba8cfa98f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:34.961528 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1176ac-44a1-43cd-8888-fcba8cfa98f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:34.962892 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1176ac-44a1-43cd-8888-fcba8cfa98f5 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:34.980074 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 538/1065] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Jul 26 13:34:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:34.981836 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64680116-21f3-44d0-9dcc-ff4ecd22746d None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:35.054290 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:35.095184 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64680116-21f3-44d0-9dcc-ff4ecd22746d None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:35.109966 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce8a184-ef21-4455-a13b-ebf5e6a49a76 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:35.110254 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce8a184-ef21-4455-a13b-ebf5e6a49a76 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:35.110533 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce8a184-ef21-4455-a13b-ebf5e6a49a76 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:35.111893 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 539/1066] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:34:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 13:34:35.117581 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03343aad-9eb3-4292-bca7-4f5665602b0f None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:35.118543 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03343aad-9eb3-4292-bca7-4f5665602b0f None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:35.119036 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03343aad-9eb3-4292-bca7-4f5665602b0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:35.131945 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d836c2-b4d7-42e0-b558-d3651ea45df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:35.132325 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d836c2-b4d7-42e0-b558-d3651ea45df0 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:35.132603 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d836c2-b4d7-42e0-b558-d3651ea45df0 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:35.132874 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d836c2-b4d7-42e0-b558-d3651ea45df0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:35.138503 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-03343aad-9eb3-4292-bca7-4f5665602b0f None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:34:35.139909 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-03343aad-9eb3-4292-bca7-4f5665602b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:34:35.140415 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d836c2-b4d7-42e0-b558-d3651ea45df0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:35.141868 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d836c2-b4d7-42e0-b558-d3651ea45df0 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:35.150465 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:35.154161 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-89d836c2-b4d7-42e0-b558-d3651ea45df0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 26 13:34:35.154161 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:35.154161 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:34:35.154161 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:34:35.154161 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:34:35.154161 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:34:35.154161 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:34:35.154161 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:34:35.154161 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:34:35.154161 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:34:35.154161 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:34:35.154161 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:34:35.154161 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 13:34:35.154161 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 13:34:35.154161 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 13:34:35.154161 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 13:34:35.155135 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:34:35.155135 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:34:35.155135 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 13:34:35.155135 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 13:34:35.155135 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 13:34:35.155135 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 13:34:35.155135 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:34:35.155135 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:34:35.155135 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:34:35.155135 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:34:35.155135 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:34:35.155135 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:34:35.155135 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:34:35.155135 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:34:35.155135 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:34:35.155135 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:34:35.155135 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:34:35.155135 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:34:35.156476 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:34:35.156476 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:34:35.156476 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 13:34:35.156476 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 13:34:35.156476 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 13:34:35.156476 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 13:34:35.156476 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 13:34:35.156476 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 13:34:35.156476 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 26 13:34:35.156476 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:34:35.156476 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 528/1067] 10.222.0.58 () {56 vars in 1093 bytes} [Tue Jul 26 13:34:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:34:35.161003 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d836c2-b4d7-42e0-b558-d3651ea45df0 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:35.217506 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 540/1068] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:34:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 13:34:35.239985 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03343aad-9eb3-4292-bca7-4f5665602b0f None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:35.269998 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d836c2-b4d7-42e0-b558-d3651ea45df0 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:35.297203 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:35.310758 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c970de-5afe-4cc4-b490-a2264d366490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:35.311381 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c970de-5afe-4cc4-b490-a2264d366490 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:35.311660 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c970de-5afe-4cc4-b490-a2264d366490 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:35.311935 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c970de-5afe-4cc4-b490-a2264d366490 None admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:35.312730 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c970de-5afe-4cc4-b490-a2264d366490 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:35.313033 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c970de-5afe-4cc4-b490-a2264d366490 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:35.314500 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c970de-5afe-4cc4-b490-a2264d366490 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:35.338408 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 529/1069] 10.222.0.58 () {56 vars in 1133 bytes} [Tue Jul 26 13:34:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:34:35.343821 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c970de-5afe-4cc4-b490-a2264d366490 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:35.355185 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03343aad-9eb3-4292-bca7-4f5665602b0f None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:35.394133 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385046d1-1595-4a8c-9033-610a874dc4e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:35.394924 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385046d1-1595-4a8c-9033-610a874dc4e0 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:35.395298 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385046d1-1595-4a8c-9033-610a874dc4e0 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:35.395740 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385046d1-1595-4a8c-9033-610a874dc4e0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:35.407397 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385046d1-1595-4a8c-9033-610a874dc4e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:34:35.412938 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-385046d1-1595-4a8c-9033-610a874dc4e0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 13:34:35.412938 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:35.412938 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:34:35.412938 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:34:35.412938 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:34:35.412938 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:34:35.412938 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:34:35.412938 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:34:35.412938 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:34:35.412938 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:34:35.412938 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:34:35.412938 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:34:35.412938 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:34:35.412938 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:34:35.412938 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:34:35.412938 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:34:35.414128 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:34:35.414128 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:34:35.414128 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:34:35.414128 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:34:35.414128 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:34:35.414128 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:34:35.414128 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:34:35.414128 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:34:35.414128 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:34:35.414128 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:34:35.414128 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:34:35.414128 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:34:35.414128 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:34:35.414128 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:34:35.414128 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:34:35.414128 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:34:35.415163 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:34:35.415163 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:34:35.415163 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:34:35.415163 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:34:35.415163 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:34:35.415163 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:34:35.415163 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:34:35.415163 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:34:35.415163 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:34:35.415163 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:34:35.415163 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 13:34:35.415163 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:34:35.415917 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 541/1070] 10.222.0.58 () {56 vars in 1096 bytes} [Tue Jul 26 13:34:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 13:34:35.420565 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385046d1-1595-4a8c-9033-610a874dc4e0 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:35.431744 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:35.459689 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c970de-5afe-4cc4-b490-a2264d366490 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:35.495608 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa51fa4-69e0-44b7-ac4b-30091be47177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:35.496158 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa51fa4-69e0-44b7-ac4b-30091be47177 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:35.496819 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa51fa4-69e0-44b7-ac4b-30091be47177 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:35.496819 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa51fa4-69e0-44b7-ac4b-30091be47177 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:35.507640 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa51fa4-69e0-44b7-ac4b-30091be47177 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:35.509066 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa51fa4-69e0-44b7-ac4b-30091be47177 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:35.524737 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-9aa51fa4-69e0-44b7-ac4b-30091be47177 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:34:35.524737 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:35.524737 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:34:35.524737 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:34:35.524737 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:34:35.524737 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:34:35.524737 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:34:35.524737 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:34:35.524737 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:34:35.524737 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:34:35.524737 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:34:35.524737 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:34:35.524737 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 13:34:35.524737 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 13:34:35.524737 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 13:34:35.525902 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 13:34:35.525902 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:34:35.525902 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:34:35.525902 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:34:35.525902 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:34:35.525902 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:34:35.525902 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:34:35.525902 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:34:35.525902 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:34:35.525902 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:34:35.525902 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:34:35.525902 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:34:35.525902 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:34:35.525902 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:34:35.525902 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:34:35.525902 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:34:35.525902 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:34:35.527077 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 13:34:35.527077 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 13:34:35.527077 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:34:35.527077 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:34:35.527077 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 530/1071] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:34:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 13:34:35.531363 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa51fa4-69e0-44b7-ac4b-30091be47177 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:35.533663 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385046d1-1595-4a8c-9033-610a874dc4e0 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:35.569194 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:35.572496 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b02b08-a55d-430d-9b01-01b7671f5d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:35.573303 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b02b08-a55d-430d-9b01-01b7671f5d2f None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:35.573625 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b02b08-a55d-430d-9b01-01b7671f5d2f None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:35.573906 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b02b08-a55d-430d-9b01-01b7671f5d2f None admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:35.574636 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b02b08-a55d-430d-9b01-01b7671f5d2f None admin] RBAC: Adding query filter params (name=admin) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:35.575036 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b02b08-a55d-430d-9b01-01b7671f5d2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:35.576408 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b02b08-a55d-430d-9b01-01b7671f5d2f None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:35.586062 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 542/1072] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Jul 26 13:34:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:34:35.591436 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b02b08-a55d-430d-9b01-01b7671f5d2f None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:35.650603 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa51fa4-69e0-44b7-ac4b-30091be47177 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:35.659134 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:35.687412 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6185c91-41cc-4da0-b853-11ccf834baed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:35.688032 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6185c91-41cc-4da0-b853-11ccf834baed None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:35.688308 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6185c91-41cc-4da0-b853-11ccf834baed None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:35.688578 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6185c91-41cc-4da0-b853-11ccf834baed None admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:35.689268 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6185c91-41cc-4da0-b853-11ccf834baed None admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:35.689600 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6185c91-41cc-4da0-b853-11ccf834baed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:35.690950 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6185c91-41cc-4da0-b853-11ccf834baed None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:35.696831 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b02b08-a55d-430d-9b01-01b7671f5d2f None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:35.708388 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 531/1073] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Jul 26 13:34:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:34:35.710037 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6185c91-41cc-4da0-b853-11ccf834baed None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:35.737230 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7471dc93-72ba-47cf-b4e8-fac7df3be34f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:35.737999 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7471dc93-72ba-47cf-b4e8-fac7df3be34f None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:35.738340 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7471dc93-72ba-47cf-b4e8-fac7df3be34f None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:35.738678 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7471dc93-72ba-47cf-b4e8-fac7df3be34f None admin] PATH_INFO: `/v3/users/95c99d354cee4bcbb241fa56ef59110c` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:35.739917 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7471dc93-72ba-47cf-b4e8-fac7df3be34f None admin] RBAC: Authorizing `identity:get_user(user_id=95c99d354cee4bcbb241fa56ef59110c)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:35.741437 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7471dc93-72ba-47cf-b4e8-fac7df3be34f None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:35.743543 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 543/1074] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Jul 26 13:34:35 2022] GET /identity/v3/users/95c99d354cee4bcbb241fa56ef59110c => generated 285 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:34:35.749088 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7471dc93-72ba-47cf-b4e8-fac7df3be34f None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:35.812895 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:35.823818 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6185c91-41cc-4da0-b853-11ccf834baed None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:35.854376 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7471dc93-72ba-47cf-b4e8-fac7df3be34f None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:35.859784 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0c1540-ed80-43b1-8a89-146cb0f60398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:35.860507 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0c1540-ed80-43b1-8a89-146cb0f60398 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:35.860896 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0c1540-ed80-43b1-8a89-146cb0f60398 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:35.861332 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0c1540-ed80-43b1-8a89-146cb0f60398 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:35.868640 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0c1540-ed80-43b1-8a89-146cb0f60398 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:35.870105 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0c1540-ed80-43b1-8a89-146cb0f60398 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:35.882664 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-af0c1540-ed80-43b1-8a89-146cb0f60398 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 13:34:35.882664 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:35.882664 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:34:35.882664 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:34:35.882664 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:34:35.882664 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:34:35.882664 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:34:35.882664 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:34:35.882664 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:34:35.882664 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:34:35.882664 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:34:35.882664 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:34:35.882664 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 13:34:35.882664 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 13:34:35.882664 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 13:34:35.883744 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 13:34:35.883744 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:34:35.883744 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:34:35.883744 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:34:35.883744 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:34:35.883744 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:34:35.883744 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:34:35.883744 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:34:35.883744 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:34:35.883744 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:34:35.883744 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:34:35.883744 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:34:35.883744 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:34:35.883744 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:34:35.883744 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:34:35.883744 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:34:35.883744 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:34:35.884639 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 13:34:35.884639 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 13:34:35.884639 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 13:34:35.884639 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 13:34:35.884639 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 13:34:35.884639 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 13:34:35.884639 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 13:34:35.884639 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:34:35.884639 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 532/1075] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Jul 26 13:34:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:34:35.889816 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0c1540-ed80-43b1-8a89-146cb0f60398 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:35.891034 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc9ffd6-dfee-4e90-875c-09a79dda52f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:35.891685 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc9ffd6-dfee-4e90-875c-09a79dda52f4 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:35.892009 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc9ffd6-dfee-4e90-875c-09a79dda52f4 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:35.892335 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc9ffd6-dfee-4e90-875c-09a79dda52f4 None admin] PATH_INFO: `/v3/projects/44d0149bb4584137bf865ded6722f055` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:35.893433 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc9ffd6-dfee-4e90-875c-09a79dda52f4 None admin] RBAC: Authorizing `identity:get_project(project_id=44d0149bb4584137bf865ded6722f055)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:35.894871 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc9ffd6-dfee-4e90-875c-09a79dda52f4 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:35.901503 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 544/1076] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Jul 26 13:34:35 2022] GET /identity/v3/projects/44d0149bb4584137bf865ded6722f055 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:35.903571 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc9ffd6-dfee-4e90-875c-09a79dda52f4 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:35.906370 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:36.006959 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0c1540-ed80-43b1-8a89-146cb0f60398 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:36.011052 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc9ffd6-dfee-4e90-875c-09a79dda52f4 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:36.048806 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e27d25-234a-4a2a-9edb-4c8ac94197db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:36.048980 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bace8a48-8695-48fb-b10f-dbb22c0598c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:36.049502 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e27d25-234a-4a2a-9edb-4c8ac94197db None admin] REQUEST_METHOD: `PUT` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:36.049608 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bace8a48-8695-48fb-b10f-dbb22c0598c7 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:36.049790 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e27d25-234a-4a2a-9edb-4c8ac94197db None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:36.049884 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bace8a48-8695-48fb-b10f-dbb22c0598c7 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:36.050065 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e27d25-234a-4a2a-9edb-4c8ac94197db None admin] PATH_INFO: `/v3/projects/44d0149bb4584137bf865ded6722f055/users/95c99d354cee4bcbb241fa56ef59110c/roles/6e0201a9a6484f7d8b112b1eafff70bd` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:36.050161 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bace8a48-8695-48fb-b10f-dbb22c0598c7 None admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:36.050838 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bace8a48-8695-48fb-b10f-dbb22c0598c7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:36.051238 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bace8a48-8695-48fb-b10f-dbb22c0598c7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:36.052340 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e27d25-234a-4a2a-9edb-4c8ac94197db None admin] RBAC: Authorizing `identity:create_grant(project_id=44d0149bb4584137bf865ded6722f055, user_id=95c99d354cee4bcbb241fa56ef59110c, role_id=6e0201a9a6484f7d8b112b1eafff70bd)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:36.052698 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bace8a48-8695-48fb-b10f-dbb22c0598c7 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:36.054618 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e27d25-234a-4a2a-9edb-4c8ac94197db None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:36.066085 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 533/1077] 10.222.0.58 () {56 vars in 1148 bytes} [Tue Jul 26 13:34:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:34:36.070298 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 545/1078] 10.222.0.58 () {58 vars in 1439 bytes} [Tue Jul 26 13:34:35 2022] PUT /identity/v3/projects/44d0149bb4584137bf865ded6722f055/users/95c99d354cee4bcbb241fa56ef59110c/roles/6e0201a9a6484f7d8b112b1eafff70bd => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 13:34:36.072943 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bace8a48-8695-48fb-b10f-dbb22c0598c7 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:36.090982 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:36.189191 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bace8a48-8695-48fb-b10f-dbb22c0598c7 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:36.230220 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e28bcd-1542-4455-abdf-c377538a15c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:36.230922 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e28bcd-1542-4455-abdf-c377538a15c1 None admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:36.231194 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e28bcd-1542-4455-abdf-c377538a15c1 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:36.231471 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e28bcd-1542-4455-abdf-c377538a15c1 None admin] PATH_INFO: `/v3/projects/1e8bb4385c204210a1bc77b1f0d654f3/users/c69161cd520b4b29bf50f3bf2f1c033b/roles/ac11ff5f40194d7d98f81a5429aa3826` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:36.233672 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e28bcd-1542-4455-abdf-c377538a15c1 None admin] RBAC: Authorizing `identity:create_grant(project_id=1e8bb4385c204210a1bc77b1f0d654f3, user_id=c69161cd520b4b29bf50f3bf2f1c033b, role_id=ac11ff5f40194d7d98f81a5429aa3826)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:36.235108 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e28bcd-1542-4455-abdf-c377538a15c1 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:36.252187 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 534/1079] 10.222.0.58 () {58 vars in 1439 bytes} [Tue Jul 26 13:34:36 2022] PUT /identity/v3/projects/1e8bb4385c204210a1bc77b1f0d654f3/users/c69161cd520b4b29bf50f3bf2f1c033b/roles/ac11ff5f40194d7d98f81a5429aa3826 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:34:36.275581 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6454ce2f-b6f7-488e-985b-b1ea5ef889c9 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:36.275836 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6454ce2f-b6f7-488e-985b-b1ea5ef889c9 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:36.276057 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6454ce2f-b6f7-488e-985b-b1ea5ef889c9 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:36.277548 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 546/1080] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:34:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:34:36.283467 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a638d986-a33b-4470-987f-d8451e1a724e None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:36.283814 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a638d986-a33b-4470-987f-d8451e1a724e None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:36.284035 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a638d986-a33b-4470-987f-d8451e1a724e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:36.304547 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-a638d986-a33b-4470-987f-d8451e1a724e None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:34:36.305855 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-a638d986-a33b-4470-987f-d8451e1a724e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:34:36.379129 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 535/1081] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:34:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 13:34:36.402317 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6454ce2f-b6f7-488e-985b-b1ea5ef889c9 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:36.472518 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:36.515972 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6454ce2f-b6f7-488e-985b-b1ea5ef889c9 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:36.551213 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615e0bec-8962-40e6-b4fa-8b6f2086675a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:36.551835 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615e0bec-8962-40e6-b4fa-8b6f2086675a None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:36.552080 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615e0bec-8962-40e6-b4fa-8b6f2086675a None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:36.552361 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615e0bec-8962-40e6-b4fa-8b6f2086675a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:36.621763 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615e0bec-8962-40e6-b4fa-8b6f2086675a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:36.623195 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615e0bec-8962-40e6-b4fa-8b6f2086675a None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:36.682945 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:36.726815 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 547/1082] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Jul 26 13:34:36 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 13:34:36.732386 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a638d986-a33b-4470-987f-d8451e1a724e None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:36.734649 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:36.834709 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a638d986-a33b-4470-987f-d8451e1a724e None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:36.868569 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d075b252-aa8c-483d-b5c6-11c1258d0292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:36.869209 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d075b252-aa8c-483d-b5c6-11c1258d0292 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:36.869524 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d075b252-aa8c-483d-b5c6-11c1258d0292 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:36.869804 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d075b252-aa8c-483d-b5c6-11c1258d0292 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:36.879900 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d075b252-aa8c-483d-b5c6-11c1258d0292 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:36.881945 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d075b252-aa8c-483d-b5c6-11c1258d0292 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:36.896710 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-d075b252-aa8c-483d-b5c6-11c1258d0292 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:34:36.896710 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:36.896710 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:34:36.896710 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:34:36.896710 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:34:36.896710 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:34:36.896710 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:34:36.896710 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:34:36.896710 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:34:36.896710 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:34:36.896710 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:34:36.896710 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:34:36.896710 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 13:34:36.896710 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 13:34:36.896710 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 13:34:36.897637 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 13:34:36.897637 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:34:36.897637 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:34:36.897637 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:34:36.897637 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:34:36.897637 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:34:36.897637 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:34:36.897637 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:34:36.897637 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:34:36.897637 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:34:36.897637 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:34:36.897637 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:34:36.897637 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:34:36.897637 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:34:36.897637 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:34:36.897637 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:34:36.897637 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:34:36.898544 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 13:34:36.898544 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 13:34:36.898544 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:34:36.898544 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:34:36.899429 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 536/1083] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:34:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:34:36.904052 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615e0bec-8962-40e6-b4fa-8b6f2086675a None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:37.039717 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615e0bec-8962-40e6-b4fa-8b6f2086675a None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:37.075801 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0799b634-f3b3-4544-a5f2-8c990449699b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:37.076424 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0799b634-f3b3-4544-a5f2-8c990449699b None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:37.076709 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0799b634-f3b3-4544-a5f2-8c990449699b None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:37.076986 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0799b634-f3b3-4544-a5f2-8c990449699b None admin] PATH_INFO: `/v3/domains` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:37.077703 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0799b634-f3b3-4544-a5f2-8c990449699b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:37.078012 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0799b634-f3b3-4544-a5f2-8c990449699b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:37.079363 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0799b634-f3b3-4544-a5f2-8c990449699b None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:37.097604 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 548/1084] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Jul 26 13:34:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:38.105355 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca413804-30eb-4961-a35f-007ca0bd72b0 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:38.105937 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca413804-30eb-4961-a35f-007ca0bd72b0 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:38.106038 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca413804-30eb-4961-a35f-007ca0bd72b0 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:38.107410 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 537/1085] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:34:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:34:38.112740 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936c3fd8-bdbc-4899-9162-e2daf42927cc None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:38.113170 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936c3fd8-bdbc-4899-9162-e2daf42927cc None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:38.113306 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936c3fd8-bdbc-4899-9162-e2daf42927cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:38.131968 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-936c3fd8-bdbc-4899-9162-e2daf42927cc None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:34:38.133165 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-936c3fd8-bdbc-4899-9162-e2daf42927cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:34:38.150210 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:38.202066 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 549/1086] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:34:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 13:34:38.225044 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca413804-30eb-4961-a35f-007ca0bd72b0 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:38.264780 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11bf454-bf2f-4782-8eb4-4ef43c2b131d None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:38.265060 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11bf454-bf2f-4782-8eb4-4ef43c2b131d None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:38.265334 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11bf454-bf2f-4782-8eb4-4ef43c2b131d None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:38.266714 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 550/1087] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:34:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:34:38.271618 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f738ee-47aa-45d6-8a7c-9f44010d2ff1 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:38.271900 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f738ee-47aa-45d6-8a7c-9f44010d2ff1 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:38.272176 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f738ee-47aa-45d6-8a7c-9f44010d2ff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:38.284750 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:38.289975 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-a2f738ee-47aa-45d6-8a7c-9f44010d2ff1 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:34:38.291102 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-a2f738ee-47aa-45d6-8a7c-9f44010d2ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:34:38.330987 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca413804-30eb-4961-a35f-007ca0bd72b0 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:38.358451 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 551/1088] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:34:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 13:34:38.364740 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509757b7-4cb4-4354-8633-3c6b234d93c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:38.365916 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509757b7-4cb4-4354-8633-3c6b234d93c7 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:38.366194 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509757b7-4cb4-4354-8633-3c6b234d93c7 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:38.366464 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509757b7-4cb4-4354-8633-3c6b234d93c7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:38.376971 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509757b7-4cb4-4354-8633-3c6b234d93c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:34:38.382299 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f738ee-47aa-45d6-8a7c-9f44010d2ff1 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:38.382426 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-509757b7-4cb4-4354-8633-3c6b234d93c7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 13:34:38.382426 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:38.382426 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:34:38.382426 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:34:38.382426 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:34:38.382426 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:34:38.382426 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:34:38.382426 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:34:38.382426 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:34:38.382426 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:34:38.382426 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:34:38.382426 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:34:38.382426 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:34:38.382426 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:34:38.382426 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:34:38.382426 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:34:38.383347 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:34:38.383347 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:34:38.383347 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:34:38.383347 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:34:38.383347 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:34:38.383347 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:34:38.383347 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:34:38.383347 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:34:38.383347 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:34:38.383347 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:34:38.383347 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:34:38.383347 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:34:38.383347 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:34:38.383347 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:34:38.383347 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:34:38.383347 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:34:38.384183 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:34:38.384183 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:34:38.384183 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:34:38.384183 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:34:38.384183 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:34:38.384183 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:34:38.384183 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:34:38.384183 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:34:38.384183 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:34:38.384183 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:34:38.384183 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 13:34:38.384183 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:34:38.384183 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 538/1089] 10.222.0.58 () {56 vars in 1096 bytes} [Tue Jul 26 13:34:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:34:38.389562 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509757b7-4cb4-4354-8633-3c6b234d93c7 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:38.473148 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:38.490905 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509757b7-4cb4-4354-8633-3c6b234d93c7 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:38.492987 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f738ee-47aa-45d6-8a7c-9f44010d2ff1 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:38.504735 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:38.525057 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030086b8-aef3-4dc0-b2fb-8301172b491a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:38.525708 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030086b8-aef3-4dc0-b2fb-8301172b491a None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:38.525990 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030086b8-aef3-4dc0-b2fb-8301172b491a None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:38.526263 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030086b8-aef3-4dc0-b2fb-8301172b491a None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:38.527089 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030086b8-aef3-4dc0-b2fb-8301172b491a None admin] RBAC: Adding query filter params (name=admin) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:38.527359 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0268695a-0985-4196-96a6-a1481b39b15a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:38.527476 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030086b8-aef3-4dc0-b2fb-8301172b491a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:38.528006 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0268695a-0985-4196-96a6-a1481b39b15a None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:38.528280 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0268695a-0985-4196-96a6-a1481b39b15a None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:38.528550 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0268695a-0985-4196-96a6-a1481b39b15a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:38.528737 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030086b8-aef3-4dc0-b2fb-8301172b491a None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:38.538349 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 539/1090] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Jul 26 13:34:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:34:38.544038 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0268695a-0985-4196-96a6-a1481b39b15a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:34:38.544180 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030086b8-aef3-4dc0-b2fb-8301172b491a None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:38.549308 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-0268695a-0985-4196-96a6-a1481b39b15a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 13:34:38.549308 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:38.549308 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:34:38.549308 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:34:38.549308 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:34:38.549308 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:34:38.549308 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:34:38.549308 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:34:38.549308 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:34:38.549308 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:34:38.549308 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:34:38.549308 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:34:38.549308 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:34:38.549308 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:34:38.549308 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:34:38.549308 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:34:38.550416 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:34:38.550416 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:34:38.550416 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:34:38.550416 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:34:38.550416 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:34:38.550416 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:34:38.550416 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:34:38.550416 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:34:38.550416 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:34:38.550416 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:34:38.550416 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:34:38.550416 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:34:38.550416 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:34:38.550416 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:34:38.550416 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:34:38.550416 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:34:38.551259 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:34:38.551259 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:34:38.551259 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:34:38.551259 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:34:38.551259 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:34:38.551259 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:34:38.551259 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:34:38.551259 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:34:38.551259 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:34:38.551259 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:34:38.551259 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 13:34:38.551259 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:34:38.551259 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 552/1091] 10.222.0.58 () {56 vars in 1120 bytes} [Tue Jul 26 13:34:38 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 13:34:38.556867 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0268695a-0985-4196-96a6-a1481b39b15a None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:38.643169 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030086b8-aef3-4dc0-b2fb-8301172b491a None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:38.654684 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0268695a-0985-4196-96a6-a1481b39b15a None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:38.677182 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbacb96-a5ae-42c0-b6c8-e110af24f457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:38.678510 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbacb96-a5ae-42c0-b6c8-e110af24f457 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:38.678801 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbacb96-a5ae-42c0-b6c8-e110af24f457 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:38.679081 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbacb96-a5ae-42c0-b6c8-e110af24f457 None admin] PATH_INFO: `/v3/users/95c99d354cee4bcbb241fa56ef59110c` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:38.680276 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbacb96-a5ae-42c0-b6c8-e110af24f457 None admin] RBAC: Authorizing `identity:get_user(user_id=95c99d354cee4bcbb241fa56ef59110c)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:38.681743 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbacb96-a5ae-42c0-b6c8-e110af24f457 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:38.683355 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 540/1092] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Jul 26 13:34:38 2022] GET /identity/v3/users/95c99d354cee4bcbb241fa56ef59110c => generated 285 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:34:38.689092 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbacb96-a5ae-42c0-b6c8-e110af24f457 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:38.689568 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beb77ed-2aad-4e3d-ab8d-1bd5070e5225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:38.690268 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beb77ed-2aad-4e3d-ab8d-1bd5070e5225 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:38.690548 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beb77ed-2aad-4e3d-ab8d-1bd5070e5225 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:38.690821 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beb77ed-2aad-4e3d-ab8d-1bd5070e5225 None admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:38.691537 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beb77ed-2aad-4e3d-ab8d-1bd5070e5225 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:38.692018 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beb77ed-2aad-4e3d-ab8d-1bd5070e5225 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:38.693380 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beb77ed-2aad-4e3d-ab8d-1bd5070e5225 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:38.710318 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 553/1093] 10.222.0.58 () {56 vars in 1115 bytes} [Tue Jul 26 13:34:38 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:38.711409 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beb77ed-2aad-4e3d-ab8d-1bd5070e5225 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:38.715784 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:38.737409 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:38.795346 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbacb96-a5ae-42c0-b6c8-e110af24f457 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:38.818974 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beb77ed-2aad-4e3d-ab8d-1bd5070e5225 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:38.830367 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b0d922-1ba8-4d52-97c6-f37f941e4b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:38.830999 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b0d922-1ba8-4d52-97c6-f37f941e4b40 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:38.831280 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b0d922-1ba8-4d52-97c6-f37f941e4b40 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:38.831556 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b0d922-1ba8-4d52-97c6-f37f941e4b40 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:38.852472 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8edeca-0b66-469d-a47c-df757b18fb0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:38.853133 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8edeca-0b66-469d-a47c-df757b18fb0f None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:38.853396 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8edeca-0b66-469d-a47c-df757b18fb0f None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:38.853697 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8edeca-0b66-469d-a47c-df757b18fb0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:38.864313 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8edeca-0b66-469d-a47c-df757b18fb0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:38.866395 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8edeca-0b66-469d-a47c-df757b18fb0f None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:38.880573 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-2e8edeca-0b66-469d-a47c-df757b18fb0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:34:38.880573 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:38.880573 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:34:38.880573 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:34:38.880573 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:34:38.880573 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:34:38.880573 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:34:38.880573 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:34:38.880573 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:34:38.880573 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:34:38.880573 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:34:38.880573 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:34:38.880573 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 13:34:38.880573 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 13:34:38.880573 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 13:34:38.881825 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 13:34:38.881825 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:34:38.881825 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:34:38.881825 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:34:38.881825 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:34:38.881825 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:34:38.881825 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:34:38.881825 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:34:38.881825 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:34:38.881825 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:34:38.881825 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:34:38.881825 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:34:38.881825 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:34:38.881825 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:34:38.881825 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:34:38.881825 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:34:38.881825 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:34:38.882916 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 13:34:38.882916 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 13:34:38.882916 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:34:38.882916 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:34:38.882916 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 554/1094] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:34:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 13:34:38.887382 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8edeca-0b66-469d-a47c-df757b18fb0f None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:38.901367 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b0d922-1ba8-4d52-97c6-f37f941e4b40 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:38.903029 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b0d922-1ba8-4d52-97c6-f37f941e4b40 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:38.948488 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:38.949163 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:38.998568 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8edeca-0b66-469d-a47c-df757b18fb0f None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:39.005501 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 541/1095] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Jul 26 13:34:38 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 26 13:34:39.012164 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b0d922-1ba8-4d52-97c6-f37f941e4b40 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:39.048859 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d87c26f-6af1-4f8f-bfbd-586ade207c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:39.049408 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d87c26f-6af1-4f8f-bfbd-586ade207c74 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:39.049726 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d87c26f-6af1-4f8f-bfbd-586ade207c74 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:39.049999 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d87c26f-6af1-4f8f-bfbd-586ade207c74 None admin] PATH_INFO: `/v3/domains` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:39.050695 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d87c26f-6af1-4f8f-bfbd-586ade207c74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:39.050998 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d87c26f-6af1-4f8f-bfbd-586ade207c74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:39.052353 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d87c26f-6af1-4f8f-bfbd-586ade207c74 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:39.064977 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 555/1096] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Jul 26 13:34:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:34:39.070839 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d87c26f-6af1-4f8f-bfbd-586ade207c74 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:39.116244 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b0d922-1ba8-4d52-97c6-f37f941e4b40 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:39.152371 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54cc28d-e631-4014-b72a-36c6b213b77c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:39.153017 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54cc28d-e631-4014-b72a-36c6b213b77c None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:39.153297 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54cc28d-e631-4014-b72a-36c6b213b77c None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:39.153605 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54cc28d-e631-4014-b72a-36c6b213b77c None admin] PATH_INFO: `/v3/projects/44d0149bb4584137bf865ded6722f055` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:39.154759 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54cc28d-e631-4014-b72a-36c6b213b77c None admin] RBAC: Authorizing `identity:get_project(project_id=44d0149bb4584137bf865ded6722f055)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:39.156236 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54cc28d-e631-4014-b72a-36c6b213b77c None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:39.157951 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 542/1097] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Jul 26 13:34:39 2022] GET /identity/v3/projects/44d0149bb4584137bf865ded6722f055 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:34:39.164568 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54cc28d-e631-4014-b72a-36c6b213b77c None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:39.176831 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d87c26f-6af1-4f8f-bfbd-586ade207c74 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:39.187858 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:39.193576 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:39.212600 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0496dfcc-a991-4e77-8e32-9fd9b28c182e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:39.213246 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0496dfcc-a991-4e77-8e32-9fd9b28c182e None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:39.213556 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0496dfcc-a991-4e77-8e32-9fd9b28c182e None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:39.213828 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0496dfcc-a991-4e77-8e32-9fd9b28c182e None admin] PATH_INFO: `/v3/users/nova` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:39.221300 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0496dfcc-a991-4e77-8e32-9fd9b28c182e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:39.222770 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0496dfcc-a991-4e77-8e32-9fd9b28c182e None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:39.235540 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-0496dfcc-a991-4e77-8e32-9fd9b28c182e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 26 13:34:39.235540 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:39.235540 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:34:39.235540 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:34:39.235540 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:34:39.235540 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:34:39.235540 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:34:39.235540 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:34:39.235540 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:34:39.235540 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:34:39.235540 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:34:39.235540 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:34:39.235540 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 13:34:39.235540 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 13:34:39.235540 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 13:34:39.235540 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 13:34:39.237384 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:34:39.237384 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:34:39.237384 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 13:34:39.237384 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 13:34:39.237384 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 13:34:39.237384 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 13:34:39.237384 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:34:39.237384 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:34:39.237384 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:34:39.237384 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:34:39.237384 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:34:39.237384 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:34:39.237384 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:34:39.237384 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:34:39.237384 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:34:39.237384 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:34:39.237384 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:34:39.237384 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:34:39.239026 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:34:39.239026 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:34:39.239026 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 13:34:39.239026 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 13:34:39.239026 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 13:34:39.239026 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 13:34:39.239026 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 13:34:39.239026 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 13:34:39.239026 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 26 13:34:39.239026 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:34:39.239026 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 556/1098] 10.222.0.58 () {56 vars in 1093 bytes} [Tue Jul 26 13:34:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 13:34:39.243726 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0496dfcc-a991-4e77-8e32-9fd9b28c182e None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:39.281327 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54cc28d-e631-4014-b72a-36c6b213b77c None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:39.318044 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac8b4fa-abbe-47a7-8156-66255f451d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:39.319116 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac8b4fa-abbe-47a7-8156-66255f451d3c None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:39.319398 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac8b4fa-abbe-47a7-8156-66255f451d3c None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:39.319669 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac8b4fa-abbe-47a7-8156-66255f451d3c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:39.320461 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac8b4fa-abbe-47a7-8156-66255f451d3c None admin] RBAC: Adding query filter params (role.id=6e0201a9a6484f7d8b112b1eafff70bd, scope.project.id=44d0149bb4584137bf865ded6722f055, user.id=95c99d354cee4bcbb241fa56ef59110c) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:39.320769 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac8b4fa-abbe-47a7-8156-66255f451d3c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:39.322221 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac8b4fa-abbe-47a7-8156-66255f451d3c None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:39.328240 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 543/1099] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Jul 26 13:34:39 2022] GET /identity/v3/role_assignments?user.id=95c99d354cee4bcbb241fa56ef59110c&scope.project.id=44d0149bb4584137bf865ded6722f055&role.id=6e0201a9a6484f7d8b112b1eafff70bd => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:34:39.356930 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0496dfcc-a991-4e77-8e32-9fd9b28c182e None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:39.394372 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f601155-c8b8-4684-a6eb-0e80c445080b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:39.395012 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f601155-c8b8-4684-a6eb-0e80c445080b None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:39.395295 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f601155-c8b8-4684-a6eb-0e80c445080b None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:39.395573 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f601155-c8b8-4684-a6eb-0e80c445080b None admin] PATH_INFO: `/v3/users` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:39.396458 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f601155-c8b8-4684-a6eb-0e80c445080b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:39.397262 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f601155-c8b8-4684-a6eb-0e80c445080b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:39.398804 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f601155-c8b8-4684-a6eb-0e80c445080b None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:39.418425 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 557/1100] 10.222.0.58 () {56 vars in 1133 bytes} [Tue Jul 26 13:34:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:34:39.424717 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac8b4fa-abbe-47a7-8156-66255f451d3c None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:39.506416 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:39.531809 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac8b4fa-abbe-47a7-8156-66255f451d3c None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:39.565157 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fbf4fa-75d4-4ae4-a6a4-ed798dd687e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:39.565811 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fbf4fa-75d4-4ae4-a6a4-ed798dd687e1 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:39.566097 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fbf4fa-75d4-4ae4-a6a4-ed798dd687e1 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:39.566375 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fbf4fa-75d4-4ae4-a6a4-ed798dd687e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:39.633336 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fbf4fa-75d4-4ae4-a6a4-ed798dd687e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:39.634926 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fbf4fa-75d4-4ae4-a6a4-ed798dd687e1 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:39.710129 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:39.733997 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 544/1101] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Jul 26 13:34:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 13:34:39.739557 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f601155-c8b8-4684-a6eb-0e80c445080b None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:39.774459 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:39.841035 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f601155-c8b8-4684-a6eb-0e80c445080b None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:39.874633 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18fc8b4-c07e-4a6d-aa5d-835ee549682c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:39.875272 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18fc8b4-c07e-4a6d-aa5d-835ee549682c None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:39.875555 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18fc8b4-c07e-4a6d-aa5d-835ee549682c None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:39.875833 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18fc8b4-c07e-4a6d-aa5d-835ee549682c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:39.886607 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18fc8b4-c07e-4a6d-aa5d-835ee549682c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:39.887974 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18fc8b4-c07e-4a6d-aa5d-835ee549682c None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:39.902384 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-d18fc8b4-c07e-4a6d-aa5d-835ee549682c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:34:39.902384 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:39.902384 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:34:39.902384 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:34:39.902384 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:34:39.902384 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:34:39.902384 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:34:39.902384 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:34:39.902384 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:34:39.902384 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:34:39.902384 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:34:39.902384 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:34:39.902384 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 13:34:39.902384 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 13:34:39.902384 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 13:34:39.903560 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 13:34:39.903560 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:34:39.903560 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:34:39.903560 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:34:39.903560 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:34:39.903560 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:34:39.903560 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:34:39.903560 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:34:39.903560 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:34:39.903560 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:34:39.903560 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:34:39.903560 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:34:39.903560 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:34:39.903560 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:34:39.903560 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:34:39.903560 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:34:39.903560 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:34:39.904928 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 13:34:39.904928 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 13:34:39.904928 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:34:39.904928 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:34:39.904928 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 558/1102] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:34:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:34:39.909292 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fbf4fa-75d4-4ae4-a6a4-ed798dd687e1 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:40.009336 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fbf4fa-75d4-4ae4-a6a4-ed798dd687e1 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:40.048465 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e075bd85-0a95-4a75-9060-af7321e64fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:40.049088 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e075bd85-0a95-4a75-9060-af7321e64fa7 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:40.049367 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e075bd85-0a95-4a75-9060-af7321e64fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:40.049667 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e075bd85-0a95-4a75-9060-af7321e64fa7 None admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:40.050355 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e075bd85-0a95-4a75-9060-af7321e64fa7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:40.050655 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e075bd85-0a95-4a75-9060-af7321e64fa7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:40.052008 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e075bd85-0a95-4a75-9060-af7321e64fa7 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:40.068233 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 545/1103] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Jul 26 13:34:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:40.069352 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18fc8b4-c07e-4a6d-aa5d-835ee549682c None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:40.152820 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:40.167952 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18fc8b4-c07e-4a6d-aa5d-835ee549682c None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:40.200246 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77124c1-3a5c-4a9b-a293-fefdf51c2db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:40.200881 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77124c1-3a5c-4a9b-a293-fefdf51c2db2 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:40.201154 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77124c1-3a5c-4a9b-a293-fefdf51c2db2 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:40.201421 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77124c1-3a5c-4a9b-a293-fefdf51c2db2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:40.208015 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77124c1-3a5c-4a9b-a293-fefdf51c2db2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:40.209455 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77124c1-3a5c-4a9b-a293-fefdf51c2db2 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:40.220471 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-d77124c1-3a5c-4a9b-a293-fefdf51c2db2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 13:34:40.220471 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:40.220471 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:34:40.220471 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:34:40.220471 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:34:40.220471 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:34:40.220471 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:34:40.220471 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:34:40.220471 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:34:40.220471 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:34:40.220471 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:34:40.220471 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:34:40.220471 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 13:34:40.220471 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 13:34:40.220471 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 13:34:40.221584 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 13:34:40.221584 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:34:40.221584 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:34:40.221584 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:34:40.221584 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:34:40.221584 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:34:40.221584 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:34:40.221584 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:34:40.221584 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:34:40.221584 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:34:40.221584 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:34:40.221584 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:34:40.221584 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:34:40.221584 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:34:40.221584 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:34:40.221584 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:34:40.221584 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:34:40.227870 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 13:34:40.227870 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 13:34:40.227870 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 13:34:40.227870 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 13:34:40.227870 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 13:34:40.227870 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 13:34:40.227870 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 13:34:40.227870 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:34:40.227870 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 559/1104] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Jul 26 13:34:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:34:40.228364 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e075bd85-0a95-4a75-9060-af7321e64fa7 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:40.263879 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:40.327450 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e075bd85-0a95-4a75-9060-af7321e64fa7 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:40.361965 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97450416-5b55-45fa-810d-05ef8eb9421e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:40.362590 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97450416-5b55-45fa-810d-05ef8eb9421e None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:40.362871 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97450416-5b55-45fa-810d-05ef8eb9421e None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:40.363140 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97450416-5b55-45fa-810d-05ef8eb9421e None admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:40.363888 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97450416-5b55-45fa-810d-05ef8eb9421e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:40.364193 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97450416-5b55-45fa-810d-05ef8eb9421e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:40.366202 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97450416-5b55-45fa-810d-05ef8eb9421e None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:40.378387 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 546/1105] 10.222.0.58 () {56 vars in 1148 bytes} [Tue Jul 26 13:34:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:40.384440 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77124c1-3a5c-4a9b-a293-fefdf51c2db2 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:40.494942 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77124c1-3a5c-4a9b-a293-fefdf51c2db2 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:40.528766 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5767124a-acd0-4abb-9e34-d0881051bbac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:40.529381 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5767124a-acd0-4abb-9e34-d0881051bbac None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:40.529691 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5767124a-acd0-4abb-9e34-d0881051bbac None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:40.529962 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5767124a-acd0-4abb-9e34-d0881051bbac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:40.530746 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5767124a-acd0-4abb-9e34-d0881051bbac None admin] RBAC: Adding query filter params (role.id=ac11ff5f40194d7d98f81a5429aa3826, scope.project.id=1e8bb4385c204210a1bc77b1f0d654f3, user.id=c69161cd520b4b29bf50f3bf2f1c033b) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:40.531051 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5767124a-acd0-4abb-9e34-d0881051bbac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:40.532452 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5767124a-acd0-4abb-9e34-d0881051bbac None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:40.538830 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 560/1106] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Jul 26 13:34:40 2022] GET /identity/v3/role_assignments?user.id=c69161cd520b4b29bf50f3bf2f1c033b&scope.project.id=1e8bb4385c204210a1bc77b1f0d654f3&role.id=ac11ff5f40194d7d98f81a5429aa3826 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:41.370863 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4d3dd3-a0a1-435e-8a24-b3b5f22b88b8 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:41.371381 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4d3dd3-a0a1-435e-8a24-b3b5f22b88b8 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:41.371538 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4d3dd3-a0a1-435e-8a24-b3b5f22b88b8 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:41.372936 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 547/1107] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:34:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:34:41.378268 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f6d94c-4766-4717-a87c-88319d8db503 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:41.378676 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f6d94c-4766-4717-a87c-88319d8db503 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:41.378827 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f6d94c-4766-4717-a87c-88319d8db503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:41.397164 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-50f6d94c-4766-4717-a87c-88319d8db503 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:34:41.398394 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-50f6d94c-4766-4717-a87c-88319d8db503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:34:41.401094 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:41.463729 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 561/1108] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:34:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 13:34:41.487770 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4d3dd3-a0a1-435e-8a24-b3b5f22b88b8 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:41.581202 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:41.587961 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4d3dd3-a0a1-435e-8a24-b3b5f22b88b8 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:41.621240 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b528ff28-d05c-4b5d-a6f9-b060f0364a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:41.621822 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b528ff28-d05c-4b5d-a6f9-b060f0364a7f None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:41.622102 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b528ff28-d05c-4b5d-a6f9-b060f0364a7f None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:41.622378 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b528ff28-d05c-4b5d-a6f9-b060f0364a7f None admin] PATH_INFO: `/v3/domains/default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:41.623519 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b528ff28-d05c-4b5d-a6f9-b060f0364a7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:41.624940 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b528ff28-d05c-4b5d-a6f9-b060f0364a7f None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:41.626562 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 548/1109] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:34:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:41.632302 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f6d94c-4766-4717-a87c-88319d8db503 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:41.731342 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f6d94c-4766-4717-a87c-88319d8db503 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:41.764384 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d7df02-0bef-42da-84ea-8e10728e3097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:41.765002 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d7df02-0bef-42da-84ea-8e10728e3097 None admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:41.765283 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d7df02-0bef-42da-84ea-8e10728e3097 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:41.765602 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d7df02-0bef-42da-84ea-8e10728e3097 None admin] PATH_INFO: `/v3/users` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:41.766275 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d7df02-0bef-42da-84ea-8e10728e3097 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:41.767685 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d7df02-0bef-42da-84ea-8e10728e3097 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:41.778799 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:41.789546 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 562/1110] 10.222.0.58 () {60 vars in 1132 bytes} [Tue Jul 26 13:34:41 2022] POST /identity/v3/users => generated 286 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:34:43.828600 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7789836a-2636-48ad-8729-b4f0c059ca34 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:43.829175 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7789836a-2636-48ad-8729-b4f0c059ca34 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:43.829175 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7789836a-2636-48ad-8729-b4f0c059ca34 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:43.830542 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 549/1111] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:34:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:34:43.835721 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5506baa-fafe-4662-bce3-1498eb69efdb None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:43.836145 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5506baa-fafe-4662-bce3-1498eb69efdb None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:43.836270 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5506baa-fafe-4662-bce3-1498eb69efdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:43.854440 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-c5506baa-fafe-4662-bce3-1498eb69efdb None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:34:43.855653 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-c5506baa-fafe-4662-bce3-1498eb69efdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:34:43.920227 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 563/1112] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:34:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 13:34:43.944073 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7789836a-2636-48ad-8729-b4f0c059ca34 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:44.046350 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7789836a-2636-48ad-8729-b4f0c059ca34 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:44.079642 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3222247c-93e0-4688-a32a-b20ee204de62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:44.080201 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3222247c-93e0-4688-a32a-b20ee204de62 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:44.080478 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3222247c-93e0-4688-a32a-b20ee204de62 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:44.080748 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3222247c-93e0-4688-a32a-b20ee204de62 None admin] PATH_INFO: `/v3/roles/5cb730ec786743c3bb8af83da90f9309` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:44.082188 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3222247c-93e0-4688-a32a-b20ee204de62 None admin] RBAC: Authorizing `identity:get_role(role_id=5cb730ec786743c3bb8af83da90f9309)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:44.083559 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3222247c-93e0-4688-a32a-b20ee204de62 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:44.084896 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 550/1113] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Jul 26 13:34:43 2022] GET /identity/v3/roles/5cb730ec786743c3bb8af83da90f9309 => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:44.090733 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5506baa-fafe-4662-bce3-1498eb69efdb None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:44.186026 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5506baa-fafe-4662-bce3-1498eb69efdb None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:44.210635 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:44.219175 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1deb270d-b152-46ee-a081-fbddcc108fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:44.219817 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deb270d-b152-46ee-a081-fbddcc108fb6 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:44.220102 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deb270d-b152-46ee-a081-fbddcc108fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:44.220374 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deb270d-b152-46ee-a081-fbddcc108fb6 None admin] PATH_INFO: `/v3/users/20e4d668f5794b1a8a6a3d85ad21addd` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:44.234587 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1deb270d-b152-46ee-a081-fbddcc108fb6 None admin] RBAC: Authorizing `identity:get_user(user_id=20e4d668f5794b1a8a6a3d85ad21addd)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:44.236003 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1deb270d-b152-46ee-a081-fbddcc108fb6 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:44.237650 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 564/1114] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Jul 26 13:34:44 2022] GET /identity/v3/users/20e4d668f5794b1a8a6a3d85ad21addd => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:44.243490 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3222247c-93e0-4688-a32a-b20ee204de62 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:44.274231 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:44.341180 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3222247c-93e0-4688-a32a-b20ee204de62 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:44.374028 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b90c091-577e-4fee-bb6b-73ae5c360774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:44.374649 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b90c091-577e-4fee-bb6b-73ae5c360774 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:44.374931 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b90c091-577e-4fee-bb6b-73ae5c360774 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:44.375203 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b90c091-577e-4fee-bb6b-73ae5c360774 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:44.440518 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b90c091-577e-4fee-bb6b-73ae5c360774 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:44.441989 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b90c091-577e-4fee-bb6b-73ae5c360774 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:44.470935 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:44.539243 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 551/1115] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Jul 26 13:34:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 13:34:44.545141 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1deb270d-b152-46ee-a081-fbddcc108fb6 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:44.641795 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1deb270d-b152-46ee-a081-fbddcc108fb6 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:44.676344 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ba8cbb-bd19-4f16-8c0d-3f951fab8b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:44.676992 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ba8cbb-bd19-4f16-8c0d-3f951fab8b71 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:44.677274 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ba8cbb-bd19-4f16-8c0d-3f951fab8b71 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:44.677587 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ba8cbb-bd19-4f16-8c0d-3f951fab8b71 None admin] PATH_INFO: `/v3/projects/44d0149bb4584137bf865ded6722f055` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:44.678749 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ba8cbb-bd19-4f16-8c0d-3f951fab8b71 None admin] RBAC: Authorizing `identity:get_project(project_id=44d0149bb4584137bf865ded6722f055)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:44.680181 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ba8cbb-bd19-4f16-8c0d-3f951fab8b71 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:44.681900 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 565/1116] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Jul 26 13:34:44 2022] GET /identity/v3/projects/44d0149bb4584137bf865ded6722f055 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:44.688156 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b90c091-577e-4fee-bb6b-73ae5c360774 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:44.794167 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b90c091-577e-4fee-bb6b-73ae5c360774 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:44.824444 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:44.830065 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317e81c8-61d5-4bff-ab14-1eef2d9f7acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:44.830716 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317e81c8-61d5-4bff-ab14-1eef2d9f7acd None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:44.830949 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317e81c8-61d5-4bff-ab14-1eef2d9f7acd None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:44.831235 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317e81c8-61d5-4bff-ab14-1eef2d9f7acd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:44.832010 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317e81c8-61d5-4bff-ab14-1eef2d9f7acd None admin] RBAC: Adding query filter params (role.id=5cb730ec786743c3bb8af83da90f9309, scope.project.id=44d0149bb4584137bf865ded6722f055, user.id=20e4d668f5794b1a8a6a3d85ad21addd) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:44.832322 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317e81c8-61d5-4bff-ab14-1eef2d9f7acd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:44.834381 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317e81c8-61d5-4bff-ab14-1eef2d9f7acd None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:44.840003 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 552/1117] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Jul 26 13:34:44 2022] GET /identity/v3/role_assignments?user.id=20e4d668f5794b1a8a6a3d85ad21addd&scope.project.id=44d0149bb4584137bf865ded6722f055&role.id=5cb730ec786743c3bb8af83da90f9309 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:46.835324 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0de6ae5-96b4-4e28-8343-612ae78dabd5 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:46.835972 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0de6ae5-96b4-4e28-8343-612ae78dabd5 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:46.836077 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0de6ae5-96b4-4e28-8343-612ae78dabd5 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:46.838102 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 566/1118] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:34:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:34:46.844331 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a244ebee-6dc1-4846-8865-f8871ea50305 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:46.844764 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a244ebee-6dc1-4846-8865-f8871ea50305 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:46.844878 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a244ebee-6dc1-4846-8865-f8871ea50305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:46.863293 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-a244ebee-6dc1-4846-8865-f8871ea50305 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:34:46.864478 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-a244ebee-6dc1-4846-8865-f8871ea50305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:34:46.930068 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 553/1119] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:34:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 13:34:46.951915 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0de6ae5-96b4-4e28-8343-612ae78dabd5 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:47.004480 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:47.052043 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0de6ae5-96b4-4e28-8343-612ae78dabd5 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:47.085144 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a238d449-6d81-4db8-921f-f950b754849b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:47.085868 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a238d449-6d81-4db8-921f-f950b754849b None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:47.086093 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a238d449-6d81-4db8-921f-f950b754849b None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:47.086366 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a238d449-6d81-4db8-921f-f950b754849b None admin] PATH_INFO: `/v3/roles/5cb730ec786743c3bb8af83da90f9309` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:47.087787 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a238d449-6d81-4db8-921f-f950b754849b None admin] RBAC: Authorizing `identity:get_role(role_id=5cb730ec786743c3bb8af83da90f9309)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:47.089144 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a238d449-6d81-4db8-921f-f950b754849b None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:47.090508 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 567/1120] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Jul 26 13:34:46 2022] GET /identity/v3/roles/5cb730ec786743c3bb8af83da90f9309 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:47.096298 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a244ebee-6dc1-4846-8865-f8871ea50305 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:47.192171 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a244ebee-6dc1-4846-8865-f8871ea50305 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:47.216888 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:47.226691 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f90efe3-ced4-4f65-9888-c31477d6abfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:47.227340 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f90efe3-ced4-4f65-9888-c31477d6abfd None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:47.227622 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f90efe3-ced4-4f65-9888-c31477d6abfd None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:47.227895 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f90efe3-ced4-4f65-9888-c31477d6abfd None admin] PATH_INFO: `/v3/users/20e4d668f5794b1a8a6a3d85ad21addd` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:47.229056 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f90efe3-ced4-4f65-9888-c31477d6abfd None admin] RBAC: Authorizing `identity:get_user(user_id=20e4d668f5794b1a8a6a3d85ad21addd)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:47.230497 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f90efe3-ced4-4f65-9888-c31477d6abfd None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:47.232088 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 554/1121] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Jul 26 13:34:47 2022] GET /identity/v3/users/20e4d668f5794b1a8a6a3d85ad21addd => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:47.237964 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a238d449-6d81-4db8-921f-f950b754849b None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:47.333308 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a238d449-6d81-4db8-921f-f950b754849b None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:47.353704 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:47.367561 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b66ccd4-dd0c-46bc-9174-1e5052586217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:47.368198 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b66ccd4-dd0c-46bc-9174-1e5052586217 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:47.368479 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b66ccd4-dd0c-46bc-9174-1e5052586217 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:47.368751 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b66ccd4-dd0c-46bc-9174-1e5052586217 None admin] PATH_INFO: `/v3/projects/44d0149bb4584137bf865ded6722f055` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:47.369923 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b66ccd4-dd0c-46bc-9174-1e5052586217 None admin] RBAC: Authorizing `identity:get_project(project_id=44d0149bb4584137bf865ded6722f055)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:47.371350 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b66ccd4-dd0c-46bc-9174-1e5052586217 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:47.373011 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 568/1122] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Jul 26 13:34:47 2022] GET /identity/v3/projects/44d0149bb4584137bf865ded6722f055 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:47.379069 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f90efe3-ced4-4f65-9888-c31477d6abfd None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:47.475339 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f90efe3-ced4-4f65-9888-c31477d6abfd None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:47.513992 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a406f980-aa04-42c4-a8c9-044ff3375a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:47.514683 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a406f980-aa04-42c4-a8c9-044ff3375a91 None admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:47.514967 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a406f980-aa04-42c4-a8c9-044ff3375a91 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:47.515244 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a406f980-aa04-42c4-a8c9-044ff3375a91 None admin] PATH_INFO: `/v3/projects/44d0149bb4584137bf865ded6722f055/users/20e4d668f5794b1a8a6a3d85ad21addd/roles/5cb730ec786743c3bb8af83da90f9309` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:47.517530 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a406f980-aa04-42c4-a8c9-044ff3375a91 None admin] RBAC: Authorizing `identity:create_grant(project_id=44d0149bb4584137bf865ded6722f055, user_id=20e4d668f5794b1a8a6a3d85ad21addd, role_id=5cb730ec786743c3bb8af83da90f9309)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:47.518953 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a406f980-aa04-42c4-a8c9-044ff3375a91 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:47.531958 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 555/1123] 10.222.0.58 () {58 vars in 1439 bytes} [Tue Jul 26 13:34:47 2022] PUT /identity/v3/projects/44d0149bb4584137bf865ded6722f055/users/20e4d668f5794b1a8a6a3d85ad21addd/roles/5cb730ec786743c3bb8af83da90f9309 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:34:49.514734 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcfb76c-4ef0-401e-bcd6-b6f7c596b3ed None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:49.515296 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcfb76c-4ef0-401e-bcd6-b6f7c596b3ed None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:49.515296 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcfb76c-4ef0-401e-bcd6-b6f7c596b3ed None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:49.516630 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 569/1124] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:34:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:34:49.521870 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502d82f0-b821-429e-9ad9-5c260c042bb2 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:49.522263 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502d82f0-b821-429e-9ad9-5c260c042bb2 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:49.522434 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502d82f0-b821-429e-9ad9-5c260c042bb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:49.539962 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-502d82f0-b821-429e-9ad9-5c260c042bb2 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:34:49.541150 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-502d82f0-b821-429e-9ad9-5c260c042bb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:34:49.589805 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:49.607487 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 556/1125] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:34:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 13:34:49.631174 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcfb76c-4ef0-401e-bcd6-b6f7c596b3ed None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:49.728471 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcfb76c-4ef0-401e-bcd6-b6f7c596b3ed None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:49.761697 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb7ecee-7a9b-4641-827c-87a438ff0306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:49.762320 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb7ecee-7a9b-4641-827c-87a438ff0306 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:49.762602 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb7ecee-7a9b-4641-827c-87a438ff0306 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:49.762877 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb7ecee-7a9b-4641-827c-87a438ff0306 None admin] PATH_INFO: `/v3/roles/5cb730ec786743c3bb8af83da90f9309` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:49.764299 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb7ecee-7a9b-4641-827c-87a438ff0306 None admin] RBAC: Authorizing `identity:get_role(role_id=5cb730ec786743c3bb8af83da90f9309)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:49.765729 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb7ecee-7a9b-4641-827c-87a438ff0306 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:49.766983 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 570/1126] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Jul 26 13:34:49 2022] GET /identity/v3/roles/5cb730ec786743c3bb8af83da90f9309 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:49.772662 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502d82f0-b821-429e-9ad9-5c260c042bb2 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:49.869070 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502d82f0-b821-429e-9ad9-5c260c042bb2 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:49.901019 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d173998e-cd2f-4c7e-9719-636dd61b18f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:49.901678 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d173998e-cd2f-4c7e-9719-636dd61b18f5 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:49.901961 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d173998e-cd2f-4c7e-9719-636dd61b18f5 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:49.902234 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d173998e-cd2f-4c7e-9719-636dd61b18f5 None admin] PATH_INFO: `/v3/users/20e4d668f5794b1a8a6a3d85ad21addd` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:49.903383 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d173998e-cd2f-4c7e-9719-636dd61b18f5 None admin] RBAC: Authorizing `identity:get_user(user_id=20e4d668f5794b1a8a6a3d85ad21addd)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:49.904794 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d173998e-cd2f-4c7e-9719-636dd61b18f5 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:49.906402 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 557/1127] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Jul 26 13:34:49 2022] GET /identity/v3/users/20e4d668f5794b1a8a6a3d85ad21addd => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:49.912658 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb7ecee-7a9b-4641-827c-87a438ff0306 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:49.966044 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:50.008097 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb7ecee-7a9b-4641-827c-87a438ff0306 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:50.041949 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3109ccc-50e8-4743-b99e-eeffb1c2dfee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:50.042572 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3109ccc-50e8-4743-b99e-eeffb1c2dfee None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:50.042856 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3109ccc-50e8-4743-b99e-eeffb1c2dfee None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:50.043130 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3109ccc-50e8-4743-b99e-eeffb1c2dfee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:50.109238 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3109ccc-50e8-4743-b99e-eeffb1c2dfee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:50.110695 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3109ccc-50e8-4743-b99e-eeffb1c2dfee None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:50.163308 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:50.206086 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 571/1128] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Jul 26 13:34:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 13:34:50.212499 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d173998e-cd2f-4c7e-9719-636dd61b18f5 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:50.264171 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:50.310614 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d173998e-cd2f-4c7e-9719-636dd61b18f5 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:50.343048 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273850e5-48b0-4a2e-a655-a8d5dc8dfec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:50.343753 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273850e5-48b0-4a2e-a655-a8d5dc8dfec7 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:50.344035 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273850e5-48b0-4a2e-a655-a8d5dc8dfec7 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:50.344307 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273850e5-48b0-4a2e-a655-a8d5dc8dfec7 None admin] PATH_INFO: `/v3/projects/44d0149bb4584137bf865ded6722f055` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:50.345433 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273850e5-48b0-4a2e-a655-a8d5dc8dfec7 None admin] RBAC: Authorizing `identity:get_project(project_id=44d0149bb4584137bf865ded6722f055)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:50.346863 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273850e5-48b0-4a2e-a655-a8d5dc8dfec7 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:50.348487 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 558/1129] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Jul 26 13:34:50 2022] GET /identity/v3/projects/44d0149bb4584137bf865ded6722f055 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:50.355056 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3109ccc-50e8-4743-b99e-eeffb1c2dfee None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:50.449057 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3109ccc-50e8-4743-b99e-eeffb1c2dfee None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:50.481983 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac121bc-45ee-44cf-a04d-1fc82da12845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:50.482586 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac121bc-45ee-44cf-a04d-1fc82da12845 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:50.482868 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac121bc-45ee-44cf-a04d-1fc82da12845 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:50.483140 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac121bc-45ee-44cf-a04d-1fc82da12845 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:50.483910 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac121bc-45ee-44cf-a04d-1fc82da12845 None admin] RBAC: Adding query filter params (role.id=5cb730ec786743c3bb8af83da90f9309, scope.project.id=44d0149bb4584137bf865ded6722f055, user.id=20e4d668f5794b1a8a6a3d85ad21addd) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:50.484217 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac121bc-45ee-44cf-a04d-1fc82da12845 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:50.485645 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac121bc-45ee-44cf-a04d-1fc82da12845 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:50.491195 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 572/1130] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Jul 26 13:34:50 2022] GET /identity/v3/role_assignments?user.id=20e4d668f5794b1a8a6a3d85ad21addd&scope.project.id=44d0149bb4584137bf865ded6722f055&role.id=5cb730ec786743c3bb8af83da90f9309 => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:52.556448 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b1282b-98f3-47c6-b810-3d582526222b None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:52.557018 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b1282b-98f3-47c6-b810-3d582526222b None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:52.557018 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b1282b-98f3-47c6-b810-3d582526222b None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:52.558356 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 559/1131] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:34:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:34:52.563969 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e11b36-c0d7-45c9-a3b2-1e752c602831 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:52.564437 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e11b36-c0d7-45c9-a3b2-1e752c602831 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:52.564530 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e11b36-c0d7-45c9-a3b2-1e752c602831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:52.583584 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-65e11b36-c0d7-45c9-a3b2-1e752c602831 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:34:52.584791 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-65e11b36-c0d7-45c9-a3b2-1e752c602831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:34:52.599497 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:52.652187 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 573/1132] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:34:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 13:34:52.675339 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b1282b-98f3-47c6-b810-3d582526222b None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:52.777812 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b1282b-98f3-47c6-b810-3d582526222b None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:52.798677 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:52.813650 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464a8d9f-aa36-43dc-9b20-aa61f98c0cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:52.814220 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464a8d9f-aa36-43dc-9b20-aa61f98c0cbc None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:52.814500 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464a8d9f-aa36-43dc-9b20-aa61f98c0cbc None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:52.814772 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464a8d9f-aa36-43dc-9b20-aa61f98c0cbc None admin] PATH_INFO: `/v3/domains/default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:52.815916 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464a8d9f-aa36-43dc-9b20-aa61f98c0cbc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:52.817334 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464a8d9f-aa36-43dc-9b20-aa61f98c0cbc None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:52.818968 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 560/1133] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:34:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:52.824940 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e11b36-c0d7-45c9-a3b2-1e752c602831 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:52.923417 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e11b36-c0d7-45c9-a3b2-1e752c602831 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:52.957057 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a44d6a-08e6-4a22-a01d-7544fde3a920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:52.957703 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a44d6a-08e6-4a22-a01d-7544fde3a920 None admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:52.957986 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a44d6a-08e6-4a22-a01d-7544fde3a920 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:52.958290 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a44d6a-08e6-4a22-a01d-7544fde3a920 None admin] PATH_INFO: `/v3/projects` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:52.958986 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a44d6a-08e6-4a22-a01d-7544fde3a920 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:52.960403 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a44d6a-08e6-4a22-a01d-7544fde3a920 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:52.987741 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 574/1134] 10.222.0.58 () {60 vars in 1141 bytes} [Tue Jul 26 13:34:52 2022] POST /identity/v3/projects => generated 311 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:34:55.001403 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f508d0-b889-4d95-8adf-130fd6baf1b2 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:55.001986 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f508d0-b889-4d95-8adf-130fd6baf1b2 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:55.001986 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f508d0-b889-4d95-8adf-130fd6baf1b2 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:55.003331 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 561/1135] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:34:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:34:55.008632 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21be46a4-4fb7-4c5f-8209-9c70185bfb8d None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:55.009025 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21be46a4-4fb7-4c5f-8209-9c70185bfb8d None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:55.009193 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21be46a4-4fb7-4c5f-8209-9c70185bfb8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:55.015516 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:55.029212 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-21be46a4-4fb7-4c5f-8209-9c70185bfb8d None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:34:55.030430 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-21be46a4-4fb7-4c5f-8209-9c70185bfb8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:34:55.095305 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 575/1136] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:34:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 13:34:55.119924 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f508d0-b889-4d95-8adf-130fd6baf1b2 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:55.218936 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f508d0-b889-4d95-8adf-130fd6baf1b2 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:55.251626 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4ecdc0-65bb-4a11-88ab-2c7c05b55081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:55.252195 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4ecdc0-65bb-4a11-88ab-2c7c05b55081 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:55.252477 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4ecdc0-65bb-4a11-88ab-2c7c05b55081 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:55.252753 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4ecdc0-65bb-4a11-88ab-2c7c05b55081 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:55.253913 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4ecdc0-65bb-4a11-88ab-2c7c05b55081 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:55.255837 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4ecdc0-65bb-4a11-88ab-2c7c05b55081 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:55.257399 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 562/1137] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:34:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:55.263057 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21be46a4-4fb7-4c5f-8209-9c70185bfb8d None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:55.357986 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21be46a4-4fb7-4c5f-8209-9c70185bfb8d None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:55.386092 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:55.391433 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df1c452-0b64-4ea4-9138-f6e0404ceb11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:55.392042 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df1c452-0b64-4ea4-9138-f6e0404ceb11 None admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:55.392325 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df1c452-0b64-4ea4-9138-f6e0404ceb11 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:55.392604 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df1c452-0b64-4ea4-9138-f6e0404ceb11 None admin] PATH_INFO: `/v3/users` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:55.393284 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df1c452-0b64-4ea4-9138-f6e0404ceb11 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:55.394718 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df1c452-0b64-4ea4-9138-f6e0404ceb11 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:55.415222 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 576/1138] 10.222.0.58 () {60 vars in 1132 bytes} [Tue Jul 26 13:34:55 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:34:57.435653 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4b4b6e-f1f4-4bd5-b21c-8ceb3ba1614c None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:57.436192 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4b4b6e-f1f4-4bd5-b21c-8ceb3ba1614c None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:57.436192 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4b4b6e-f1f4-4bd5-b21c-8ceb3ba1614c None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:57.437661 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 563/1139] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:34:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:34:57.443009 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446aa34c-68e9-40a6-a015-88f18313cd56 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:57.443465 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446aa34c-68e9-40a6-a015-88f18313cd56 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:57.443559 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446aa34c-68e9-40a6-a015-88f18313cd56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:57.462264 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-446aa34c-68e9-40a6-a015-88f18313cd56 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:34:57.463467 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-446aa34c-68e9-40a6-a015-88f18313cd56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:34:57.531063 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 577/1140] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:34:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 13:34:57.554839 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4b4b6e-f1f4-4bd5-b21c-8ceb3ba1614c None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:57.611792 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:57.656451 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4b4b6e-f1f4-4bd5-b21c-8ceb3ba1614c None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:57.690781 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc5c6cc-a125-4f6f-8cb8-13968f8d4c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:57.691334 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc5c6cc-a125-4f6f-8cb8-13968f8d4c9d None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:57.691619 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc5c6cc-a125-4f6f-8cb8-13968f8d4c9d None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:57.691892 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc5c6cc-a125-4f6f-8cb8-13968f8d4c9d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:57.702128 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc5c6cc-a125-4f6f-8cb8-13968f8d4c9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:34:57.707032 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-2fc5c6cc-a125-4f6f-8cb8-13968f8d4c9d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 13:34:57.707032 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:34:57.707032 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:34:57.707032 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:34:57.707032 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:34:57.707032 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:34:57.707032 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:34:57.707032 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:34:57.707032 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:34:57.707032 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:34:57.707032 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:34:57.707032 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:34:57.707032 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:34:57.707032 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:34:57.707032 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:34:57.707032 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:34:57.708677 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:34:57.708677 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:34:57.708677 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:34:57.708677 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:34:57.708677 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:34:57.708677 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:34:57.708677 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:34:57.708677 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:34:57.708677 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:34:57.708677 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:34:57.708677 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:34:57.708677 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:34:57.708677 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:34:57.708677 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:34:57.708677 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:34:57.708677 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:34:57.709974 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:34:57.709974 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:34:57.709974 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:34:57.709974 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:34:57.709974 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:34:57.709974 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:34:57.709974 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:34:57.709974 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:34:57.709974 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:34:57.709974 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:34:57.709974 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 13:34:57.709974 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:34:57.709974 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 564/1141] 10.222.0.58 () {56 vars in 1096 bytes} [Tue Jul 26 13:34:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:34:57.714767 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446aa34c-68e9-40a6-a015-88f18313cd56 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:57.810367 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446aa34c-68e9-40a6-a015-88f18313cd56 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:57.827801 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:57.847207 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b6c705-db8c-459a-9035-594d00a0a942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:57.847834 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b6c705-db8c-459a-9035-594d00a0a942 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:57.848112 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b6c705-db8c-459a-9035-594d00a0a942 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:57.848409 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b6c705-db8c-459a-9035-594d00a0a942 None admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:57.849698 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b6c705-db8c-459a-9035-594d00a0a942 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:57.850005 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b6c705-db8c-459a-9035-594d00a0a942 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:57.851406 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b6c705-db8c-459a-9035-594d00a0a942 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:57.860390 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 578/1142] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Jul 26 13:34:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:57.866598 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc5c6cc-a125-4f6f-8cb8-13968f8d4c9d None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:57.965453 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc5c6cc-a125-4f6f-8cb8-13968f8d4c9d None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:57.981744 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:57.999160 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc45d86-87cd-4499-9af8-9b54794c64d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:57.999793 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc45d86-87cd-4499-9af8-9b54794c64d2 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:58.000078 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc45d86-87cd-4499-9af8-9b54794c64d2 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:58.000355 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc45d86-87cd-4499-9af8-9b54794c64d2 None admin] PATH_INFO: `/v3/users/be96631b57214572906c9579be278f88` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:58.013124 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc45d86-87cd-4499-9af8-9b54794c64d2 None admin] RBAC: Authorizing `identity:get_user(user_id=be96631b57214572906c9579be278f88)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:58.014592 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc45d86-87cd-4499-9af8-9b54794c64d2 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:58.016195 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 565/1143] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Jul 26 13:34:57 2022] GET /identity/v3/users/be96631b57214572906c9579be278f88 => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:58.022681 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b6c705-db8c-459a-9035-594d00a0a942 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:58.122292 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b6c705-db8c-459a-9035-594d00a0a942 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:58.155214 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8784e4-307c-4ca6-9329-222c5070d6a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:58.155830 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8784e4-307c-4ca6-9329-222c5070d6a3 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:58.156115 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8784e4-307c-4ca6-9329-222c5070d6a3 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:58.156393 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8784e4-307c-4ca6-9329-222c5070d6a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:58.203900 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:58.225538 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8784e4-307c-4ca6-9329-222c5070d6a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:58.226947 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8784e4-307c-4ca6-9329-222c5070d6a3 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:58.324787 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 579/1144] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Jul 26 13:34:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 13:34:58.330717 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc45d86-87cd-4499-9af8-9b54794c64d2 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:58.428634 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc45d86-87cd-4499-9af8-9b54794c64d2 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:58.461663 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f37b5ba-13ea-4194-ae8a-df9e1b9b9ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:58.462302 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f37b5ba-13ea-4194-ae8a-df9e1b9b9ac1 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:58.462582 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f37b5ba-13ea-4194-ae8a-df9e1b9b9ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:58.462858 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f37b5ba-13ea-4194-ae8a-df9e1b9b9ac1 None admin] PATH_INFO: `/v3/projects/d3deb6c135d343009f7a253c50760643` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:58.464009 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f37b5ba-13ea-4194-ae8a-df9e1b9b9ac1 None admin] RBAC: Authorizing `identity:get_project(project_id=d3deb6c135d343009f7a253c50760643)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:58.465428 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f37b5ba-13ea-4194-ae8a-df9e1b9b9ac1 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:58.467121 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 566/1145] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Jul 26 13:34:58 2022] GET /identity/v3/projects/d3deb6c135d343009f7a253c50760643 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:34:58.473225 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8784e4-307c-4ca6-9329-222c5070d6a3 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:34:58.551056 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:34:58.573293 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8784e4-307c-4ca6-9329-222c5070d6a3 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:34:58.606368 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32580cb4-32e4-40d3-8392-6dcb80a2fd29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:34:58.606984 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32580cb4-32e4-40d3-8392-6dcb80a2fd29 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:34:58.607269 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32580cb4-32e4-40d3-8392-6dcb80a2fd29 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:34:58.607549 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32580cb4-32e4-40d3-8392-6dcb80a2fd29 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:34:58.608336 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32580cb4-32e4-40d3-8392-6dcb80a2fd29 None admin] RBAC: Adding query filter params (role.id=6e0201a9a6484f7d8b112b1eafff70bd, scope.project.id=d3deb6c135d343009f7a253c50760643, user.id=be96631b57214572906c9579be278f88) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:34:58.608649 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32580cb4-32e4-40d3-8392-6dcb80a2fd29 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:34:58.610108 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32580cb4-32e4-40d3-8392-6dcb80a2fd29 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:34:58.615953 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 580/1146] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Jul 26 13:34:58 2022] GET /identity/v3/role_assignments?user.id=be96631b57214572906c9579be278f88&scope.project.id=d3deb6c135d343009f7a253c50760643&role.id=6e0201a9a6484f7d8b112b1eafff70bd => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:35:00.585103 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4bc11e-3826-4e73-9554-4790a4b26f87 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:35:00.585717 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4bc11e-3826-4e73-9554-4790a4b26f87 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:35:00.585717 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4bc11e-3826-4e73-9554-4790a4b26f87 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:35:00.587049 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 567/1147] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:35:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:35:00.593354 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3f8f1a-dd49-4f2e-8724-b5a1ffd09e4f None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:35:00.593858 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3f8f1a-dd49-4f2e-8724-b5a1ffd09e4f None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:35:00.593973 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3f8f1a-dd49-4f2e-8724-b5a1ffd09e4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:35:00.612386 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-5d3f8f1a-dd49-4f2e-8724-b5a1ffd09e4f None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:35:00.613638 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-5d3f8f1a-dd49-4f2e-8724-b5a1ffd09e4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:35:00.681285 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 581/1148] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:35:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 13:35:00.703168 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4bc11e-3826-4e73-9554-4790a4b26f87 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:35:00.747994 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:35:00.805570 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4bc11e-3826-4e73-9554-4790a4b26f87 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:35:00.838827 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46115a46-9d55-4a85-bbed-b6be7de13837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:35:00.839383 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46115a46-9d55-4a85-bbed-b6be7de13837 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:35:00.839661 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46115a46-9d55-4a85-bbed-b6be7de13837 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:35:00.839934 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46115a46-9d55-4a85-bbed-b6be7de13837 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:35:00.850391 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46115a46-9d55-4a85-bbed-b6be7de13837 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:35:00.855171 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-46115a46-9d55-4a85-bbed-b6be7de13837 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 13:35:00.855171 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:35:00.855171 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:35:00.855171 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:35:00.855171 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:35:00.855171 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:35:00.855171 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:35:00.855171 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:35:00.855171 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:35:00.855171 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:35:00.855171 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:35:00.855171 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:35:00.855171 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:35:00.855171 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:35:00.855171 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:35:00.855171 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:35:00.856261 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:35:00.856261 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:35:00.856261 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:35:00.856261 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:35:00.856261 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:35:00.856261 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:35:00.856261 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:35:00.856261 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:35:00.856261 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:35:00.856261 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:35:00.856261 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:35:00.856261 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:35:00.856261 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:35:00.856261 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:35:00.856261 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:35:00.856261 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:35:00.857415 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:35:00.857415 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:35:00.857415 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:35:00.857415 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:35:00.857415 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:35:00.857415 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:35:00.857415 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:35:00.857415 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:35:00.857415 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:35:00.857415 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:35:00.857415 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 13:35:00.857415 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:35:00.857415 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 568/1149] 10.222.0.58 () {56 vars in 1096 bytes} [Tue Jul 26 13:35:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:35:00.862348 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3f8f1a-dd49-4f2e-8724-b5a1ffd09e4f None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:35:00.935305 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:35:00.960364 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3f8f1a-dd49-4f2e-8724-b5a1ffd09e4f None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:35:00.994360 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f370761c-9299-4b0d-9a4e-ef684a9b449d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:35:00.994993 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f370761c-9299-4b0d-9a4e-ef684a9b449d None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:35:00.995279 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f370761c-9299-4b0d-9a4e-ef684a9b449d None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:35:00.995556 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f370761c-9299-4b0d-9a4e-ef684a9b449d None admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:35:00.996277 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f370761c-9299-4b0d-9a4e-ef684a9b449d None admin] RBAC: Adding query filter params (name=admin) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:35:00.996586 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f370761c-9299-4b0d-9a4e-ef684a9b449d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:35:00.997987 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f370761c-9299-4b0d-9a4e-ef684a9b449d None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:35:01.006211 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 582/1150] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Jul 26 13:35:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:35:01.012096 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46115a46-9d55-4a85-bbed-b6be7de13837 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:35:01.113065 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46115a46-9d55-4a85-bbed-b6be7de13837 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:35:01.114519 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:35:01.147887 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439c1c02-f0b8-497f-8067-2589004b94dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:35:01.148515 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439c1c02-f0b8-497f-8067-2589004b94dc None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:35:01.148792 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439c1c02-f0b8-497f-8067-2589004b94dc None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:35:01.149063 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439c1c02-f0b8-497f-8067-2589004b94dc None admin] PATH_INFO: `/v3/users/be96631b57214572906c9579be278f88` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:35:01.150231 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439c1c02-f0b8-497f-8067-2589004b94dc None admin] RBAC: Authorizing `identity:get_user(user_id=be96631b57214572906c9579be278f88)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:35:01.151643 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439c1c02-f0b8-497f-8067-2589004b94dc None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:35:01.153217 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 569/1151] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Jul 26 13:35:01 2022] GET /identity/v3/users/be96631b57214572906c9579be278f88 => generated 286 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:35:01.159051 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f370761c-9299-4b0d-9a4e-ef684a9b449d None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:35:01.256103 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f370761c-9299-4b0d-9a4e-ef684a9b449d None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:35:01.288082 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785f4b99-90c5-4285-b15d-af6bc0834c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:35:01.288703 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785f4b99-90c5-4285-b15d-af6bc0834c5a None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:35:01.288982 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785f4b99-90c5-4285-b15d-af6bc0834c5a None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:35:01.289254 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785f4b99-90c5-4285-b15d-af6bc0834c5a None admin] PATH_INFO: `/v3/projects/d3deb6c135d343009f7a253c50760643` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:35:01.290406 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785f4b99-90c5-4285-b15d-af6bc0834c5a None admin] RBAC: Authorizing `identity:get_project(project_id=d3deb6c135d343009f7a253c50760643)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:35:01.291810 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785f4b99-90c5-4285-b15d-af6bc0834c5a None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:35:01.293461 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 583/1152] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Jul 26 13:35:01 2022] GET /identity/v3/projects/d3deb6c135d343009f7a253c50760643 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:35:01.299540 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439c1c02-f0b8-497f-8067-2589004b94dc None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:35:01.395903 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439c1c02-f0b8-497f-8067-2589004b94dc None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:35:01.429087 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de5e0b8-eecd-4194-b7b4-9e798d1c8a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:35:01.429764 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de5e0b8-eecd-4194-b7b4-9e798d1c8a23 None admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:35:01.430047 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de5e0b8-eecd-4194-b7b4-9e798d1c8a23 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:35:01.430323 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de5e0b8-eecd-4194-b7b4-9e798d1c8a23 None admin] PATH_INFO: `/v3/projects/d3deb6c135d343009f7a253c50760643/users/be96631b57214572906c9579be278f88/roles/6e0201a9a6484f7d8b112b1eafff70bd` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:35:01.432216 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de5e0b8-eecd-4194-b7b4-9e798d1c8a23 None admin] RBAC: Authorizing `identity:create_grant(project_id=d3deb6c135d343009f7a253c50760643, user_id=be96631b57214572906c9579be278f88, role_id=6e0201a9a6484f7d8b112b1eafff70bd)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:35:01.433650 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de5e0b8-eecd-4194-b7b4-9e798d1c8a23 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:35:01.446883 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 570/1153] 10.222.0.58 () {58 vars in 1439 bytes} [Tue Jul 26 13:35:01 2022] PUT /identity/v3/projects/d3deb6c135d343009f7a253c50760643/users/be96631b57214572906c9579be278f88/roles/6e0201a9a6484f7d8b112b1eafff70bd => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:35:03.420284 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0fb044-d498-49f0-b793-bfde48f6e35b None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:35:03.420846 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0fb044-d498-49f0-b793-bfde48f6e35b None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:35:03.420846 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0fb044-d498-49f0-b793-bfde48f6e35b None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:35:03.422308 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 584/1154] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:35:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:35:03.428251 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531d36ab-2eb1-42d4-bf8e-695bc5509f32 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:35:03.429031 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531d36ab-2eb1-42d4-bf8e-695bc5509f32 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:35:03.429304 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531d36ab-2eb1-42d4-bf8e-695bc5509f32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:35:03.447078 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-531d36ab-2eb1-42d4-bf8e-695bc5509f32 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:35:03.448269 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-531d36ab-2eb1-42d4-bf8e-695bc5509f32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:35:03.471965 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:35:03.515442 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 571/1155] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:35:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 13:35:03.538712 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0fb044-d498-49f0-b793-bfde48f6e35b None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:35:03.542522 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:35:03.638234 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0fb044-d498-49f0-b793-bfde48f6e35b None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:35:03.671235 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52438dd1-8590-4abf-a523-b6cb602fc8e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:35:03.671796 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52438dd1-8590-4abf-a523-b6cb602fc8e5 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:35:03.672077 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52438dd1-8590-4abf-a523-b6cb602fc8e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:35:03.672349 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52438dd1-8590-4abf-a523-b6cb602fc8e5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:35:03.683430 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52438dd1-8590-4abf-a523-b6cb602fc8e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:35:03.688344 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-52438dd1-8590-4abf-a523-b6cb602fc8e5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 13:35:03.688344 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:35:03.688344 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:35:03.688344 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:35:03.688344 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:35:03.688344 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:35:03.688344 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:35:03.688344 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:35:03.688344 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:35:03.688344 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:35:03.688344 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:35:03.688344 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:35:03.688344 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:35:03.688344 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:35:03.688344 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:35:03.688344 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:35:03.689458 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:35:03.689458 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:35:03.689458 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:35:03.689458 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:35:03.689458 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:35:03.689458 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:35:03.689458 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:35:03.689458 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:35:03.689458 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:35:03.689458 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:35:03.689458 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:35:03.689458 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:35:03.689458 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:35:03.689458 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:35:03.689458 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:35:03.689458 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:35:03.690545 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:35:03.690545 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:35:03.690545 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:35:03.690545 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:35:03.690545 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:35:03.690545 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:35:03.690545 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:35:03.690545 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:35:03.690545 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:35:03.690545 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:35:03.690545 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 13:35:03.690545 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:35:03.690545 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 585/1156] 10.222.0.58 () {56 vars in 1096 bytes} [Tue Jul 26 13:35:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:35:03.695888 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531d36ab-2eb1-42d4-bf8e-695bc5509f32 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:35:03.793984 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531d36ab-2eb1-42d4-bf8e-695bc5509f32 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:35:03.827773 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbffdfaf-adf3-434e-8658-893fb64c361c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:35:03.828382 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbffdfaf-adf3-434e-8658-893fb64c361c None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:35:03.828684 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbffdfaf-adf3-434e-8658-893fb64c361c None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:35:03.828954 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbffdfaf-adf3-434e-8658-893fb64c361c None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:35:03.829687 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbffdfaf-adf3-434e-8658-893fb64c361c None admin] RBAC: Adding query filter params (name=admin) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:35:03.829990 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbffdfaf-adf3-434e-8658-893fb64c361c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:35:03.831355 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbffdfaf-adf3-434e-8658-893fb64c361c None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:35:03.839641 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 572/1157] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Jul 26 13:35:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:35:03.845558 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52438dd1-8590-4abf-a523-b6cb602fc8e5 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:35:03.905878 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:35:03.943665 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52438dd1-8590-4abf-a523-b6cb602fc8e5 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:35:03.976322 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e461c00c-ca1e-41bf-b95f-fcf557acd0b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:35:03.976954 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e461c00c-ca1e-41bf-b95f-fcf557acd0b0 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:35:03.977234 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e461c00c-ca1e-41bf-b95f-fcf557acd0b0 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:35:03.977531 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e461c00c-ca1e-41bf-b95f-fcf557acd0b0 None admin] PATH_INFO: `/v3/users/be96631b57214572906c9579be278f88` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:35:03.978774 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e461c00c-ca1e-41bf-b95f-fcf557acd0b0 None admin] RBAC: Authorizing `identity:get_user(user_id=be96631b57214572906c9579be278f88)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:35:03.980213 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e461c00c-ca1e-41bf-b95f-fcf557acd0b0 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:35:03.981860 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 586/1158] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Jul 26 13:35:03 2022] GET /identity/v3/users/be96631b57214572906c9579be278f88 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:35:03.987927 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbffdfaf-adf3-434e-8658-893fb64c361c None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:35:04.015044 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:35:04.094186 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbffdfaf-adf3-434e-8658-893fb64c361c None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:35:04.126382 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e058f3-db42-4b64-b8e4-7ce0febc9369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:35:04.127002 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e058f3-db42-4b64-b8e4-7ce0febc9369 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:35:04.127287 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e058f3-db42-4b64-b8e4-7ce0febc9369 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:35:04.127566 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e058f3-db42-4b64-b8e4-7ce0febc9369 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:35:04.193495 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e058f3-db42-4b64-b8e4-7ce0febc9369 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:35:04.195005 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e058f3-db42-4b64-b8e4-7ce0febc9369 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:35:04.218660 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:35:04.292157 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 573/1159] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Jul 26 13:35:03 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 13:35:04.298326 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e461c00c-ca1e-41bf-b95f-fcf557acd0b0 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:35:04.393648 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e461c00c-ca1e-41bf-b95f-fcf557acd0b0 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:35:04.421989 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:35:04.427847 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b00a35f-5859-4692-833f-fc7b9d019b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:35:04.428510 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b00a35f-5859-4692-833f-fc7b9d019b6d None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:35:04.428819 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b00a35f-5859-4692-833f-fc7b9d019b6d None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:35:04.429096 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b00a35f-5859-4692-833f-fc7b9d019b6d None admin] PATH_INFO: `/v3/projects/d3deb6c135d343009f7a253c50760643` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:35:04.430377 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b00a35f-5859-4692-833f-fc7b9d019b6d None admin] RBAC: Authorizing `identity:get_project(project_id=d3deb6c135d343009f7a253c50760643)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:35:04.431824 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b00a35f-5859-4692-833f-fc7b9d019b6d None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:35:04.433552 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 587/1160] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Jul 26 13:35:04 2022] GET /identity/v3/projects/d3deb6c135d343009f7a253c50760643 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:35:04.439754 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e058f3-db42-4b64-b8e4-7ce0febc9369 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:35:04.536969 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e058f3-db42-4b64-b8e4-7ce0febc9369 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:35:04.559379 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:35:04.571522 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f389650a-4d42-475c-a732-d2342fba0a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:35:04.572135 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f389650a-4d42-475c-a732-d2342fba0a7e None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:35:04.572415 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f389650a-4d42-475c-a732-d2342fba0a7e None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:35:04.572688 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f389650a-4d42-475c-a732-d2342fba0a7e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:35:04.573492 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f389650a-4d42-475c-a732-d2342fba0a7e None admin] RBAC: Adding query filter params (role.id=6e0201a9a6484f7d8b112b1eafff70bd, scope.project.id=d3deb6c135d343009f7a253c50760643, user.id=be96631b57214572906c9579be278f88) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:35:04.573813 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f389650a-4d42-475c-a732-d2342fba0a7e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:35:04.575230 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f389650a-4d42-475c-a732-d2342fba0a7e None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:35:04.580982 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 574/1161] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Jul 26 13:35:04 2022] GET /identity/v3/role_assignments?user.id=be96631b57214572906c9579be278f88&scope.project.id=d3deb6c135d343009f7a253c50760643&role.id=6e0201a9a6484f7d8b112b1eafff70bd => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:35:06.594070 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac550b9-ed7e-45d6-a241-cf7c21c8076b None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:35:06.594654 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac550b9-ed7e-45d6-a241-cf7c21c8076b None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:35:06.594654 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac550b9-ed7e-45d6-a241-cf7c21c8076b None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:35:06.595977 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 588/1162] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:35:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:35:06.602859 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21798ad4-c5be-40f4-9946-7dc730d730b7 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:35:06.603357 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21798ad4-c5be-40f4-9946-7dc730d730b7 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:35:06.603759 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21798ad4-c5be-40f4-9946-7dc730d730b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:35:06.622447 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-21798ad4-c5be-40f4-9946-7dc730d730b7 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:35:06.623671 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-21798ad4-c5be-40f4-9946-7dc730d730b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:35:06.695078 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 575/1163] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:35:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 13:35:06.720177 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac550b9-ed7e-45d6-a241-cf7c21c8076b None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:35:06.823912 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac550b9-ed7e-45d6-a241-cf7c21c8076b None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:35:06.858414 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7ec4ed-20f0-4d35-85b0-5cb28cbc506b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:35:06.858950 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7ec4ed-20f0-4d35-85b0-5cb28cbc506b None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:35:06.859232 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7ec4ed-20f0-4d35-85b0-5cb28cbc506b None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:35:06.859508 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7ec4ed-20f0-4d35-85b0-5cb28cbc506b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:35:06.924336 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:35:06.926646 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7ec4ed-20f0-4d35-85b0-5cb28cbc506b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:35:06.928383 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7ec4ed-20f0-4d35-85b0-5cb28cbc506b None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:35:07.028701 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 589/1164] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Jul 26 13:35:06 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 13:35:07.035005 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21798ad4-c5be-40f4-9946-7dc730d730b7 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:35:07.134274 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21798ad4-c5be-40f4-9946-7dc730d730b7 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:35:07.150585 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:35:07.167770 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d124f2c5-398d-4268-981f-6061249d7550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:35:07.168407 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d124f2c5-398d-4268-981f-6061249d7550 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:35:07.168688 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d124f2c5-398d-4268-981f-6061249d7550 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:35:07.168960 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d124f2c5-398d-4268-981f-6061249d7550 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:35:07.175715 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d124f2c5-398d-4268-981f-6061249d7550 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:35:07.177175 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d124f2c5-398d-4268-981f-6061249d7550 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:35:07.189022 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-d124f2c5-398d-4268-981f-6061249d7550 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 26 13:35:07.189022 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:35:07.189022 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 26 13:35:07.189022 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 26 13:35:07.189022 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 13:35:07.189022 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 13:35:07.189022 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 13:35:07.189022 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 13:35:07.189022 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 26 13:35:07.189022 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:35:07.189022 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 13:35:07.189022 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:35:07.189022 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:35:07.189022 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:35:07.189022 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:35:07.189022 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:35:07.189022 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:35:07.190285 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:35:07.190285 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:35:07.190285 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:35:07.190285 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:35:07.190285 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:35:07.190285 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:35:07.190285 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 13:35:07.190285 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 13:35:07.190285 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 13:35:07.190285 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 13:35:07.190285 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:35:07.190285 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:35:07.190285 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:35:07.190285 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:35:07.190285 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:35:07.190285 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:35:07.191255 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:35:07.191255 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:35:07.191255 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:35:07.191255 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:35:07.191255 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:35:07.191255 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:35:07.191255 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:35:07.191255 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:35:07.191255 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:35:07.191255 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:35:07.191255 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 26 13:35:07.191255 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 13:35:07.191255 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 26 13:35:07.191255 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:35:07.191255 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 576/1165] 10.222.0.58 () {56 vars in 1117 bytes} [Tue Jul 26 13:35:07 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:35:07.197105 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7ec4ed-20f0-4d35-85b0-5cb28cbc506b None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:35:07.292956 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:35:07.296178 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7ec4ed-20f0-4d35-85b0-5cb28cbc506b None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:35:07.329526 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d070c8fb-68cb-4bbc-bd09-48d4f7ae50fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:35:07.330142 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d070c8fb-68cb-4bbc-bd09-48d4f7ae50fd None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:35:07.330423 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d070c8fb-68cb-4bbc-bd09-48d4f7ae50fd None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:35:07.330698 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d070c8fb-68cb-4bbc-bd09-48d4f7ae50fd None admin] PATH_INFO: `/v3/domains` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:35:07.331380 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d070c8fb-68cb-4bbc-bd09-48d4f7ae50fd None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:35:07.331685 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d070c8fb-68cb-4bbc-bd09-48d4f7ae50fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:35:07.333610 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d070c8fb-68cb-4bbc-bd09-48d4f7ae50fd None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:35:07.340642 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 590/1166] 10.222.0.58 () {56 vars in 1115 bytes} [Tue Jul 26 13:35:07 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:35:09.315960 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f781f8-057a-4b33-9888-6b4bfe113df5 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:35:09.316458 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f781f8-057a-4b33-9888-6b4bfe113df5 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:35:09.316458 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f781f8-057a-4b33-9888-6b4bfe113df5 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:35:09.317868 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 577/1167] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:35:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:35:09.322996 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fc4fe9-fcbd-488a-964b-7d610caddd65 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:35:09.323386 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fc4fe9-fcbd-488a-964b-7d610caddd65 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:35:09.323552 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fc4fe9-fcbd-488a-964b-7d610caddd65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:35:09.341242 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-e1fc4fe9-fcbd-488a-964b-7d610caddd65 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:35:09.342456 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-e1fc4fe9-fcbd-488a-964b-7d610caddd65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:35:09.406992 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 591/1168] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:35:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 13:35:09.430534 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f781f8-057a-4b33-9888-6b4bfe113df5 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:35:09.527917 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f781f8-057a-4b33-9888-6b4bfe113df5 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:35:09.561907 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13877058-18b1-43ba-8722-1378d13aeeda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:35:09.562504 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13877058-18b1-43ba-8722-1378d13aeeda None admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:35:09.562781 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13877058-18b1-43ba-8722-1378d13aeeda None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:35:09.563052 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13877058-18b1-43ba-8722-1378d13aeeda None admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:35:09.563750 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13877058-18b1-43ba-8722-1378d13aeeda None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:35:09.565091 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13877058-18b1-43ba-8722-1378d13aeeda None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:35:09.580367 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 578/1169] 10.222.0.58 () {60 vars in 1138 bytes} [Tue Jul 26 13:35:09 2022] POST /identity/v3/domains => generated 267 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 13:35:11.597352 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b482070-88c4-4ea5-bd8c-7b2fc961c0ac None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:35:11.597908 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b482070-88c4-4ea5-bd8c-7b2fc961c0ac None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:35:11.597998 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b482070-88c4-4ea5-bd8c-7b2fc961c0ac None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:35:11.599295 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 592/1170] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:35:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:35:11.604504 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5783cc4a-1cde-445e-b214-d99ac4d48e5c None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:35:11.604927 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5783cc4a-1cde-445e-b214-d99ac4d48e5c None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:35:11.605047 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5783cc4a-1cde-445e-b214-d99ac4d48e5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:35:11.622585 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-5783cc4a-1cde-445e-b214-d99ac4d48e5c None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:35:11.623767 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-5783cc4a-1cde-445e-b214-d99ac4d48e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:35:11.652189 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:35:11.692933 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 579/1171] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:35:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 13:35:11.716705 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b482070-88c4-4ea5-bd8c-7b2fc961c0ac None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:35:11.809755 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:35:11.818303 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b482070-88c4-4ea5-bd8c-7b2fc961c0ac None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:35:11.853068 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f08d63d-400d-45de-b67f-63b10c7a9e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:35:11.853671 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f08d63d-400d-45de-b67f-63b10c7a9e6b None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:35:11.853956 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f08d63d-400d-45de-b67f-63b10c7a9e6b None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:35:11.854233 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f08d63d-400d-45de-b67f-63b10c7a9e6b None admin] PATH_INFO: `/v3/domains/a35a8059601c455b9eca35e324840487` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:35:11.865768 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f08d63d-400d-45de-b67f-63b10c7a9e6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=a35a8059601c455b9eca35e324840487)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:35:11.867222 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f08d63d-400d-45de-b67f-63b10c7a9e6b None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:35:11.868914 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 593/1172] 10.222.0.58 () {56 vars in 1183 bytes} [Tue Jul 26 13:35:11 2022] GET /identity/v3/domains/a35a8059601c455b9eca35e324840487 => generated 267 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:35:11.874873 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5783cc4a-1cde-445e-b214-d99ac4d48e5c None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:35:11.979147 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5783cc4a-1cde-445e-b214-d99ac4d48e5c None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:35:12.013284 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24d64c3-1913-4b87-9f5a-15bc12758245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:35:12.013931 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24d64c3-1913-4b87-9f5a-15bc12758245 None admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:35:12.014215 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24d64c3-1913-4b87-9f5a-15bc12758245 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:35:12.014491 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24d64c3-1913-4b87-9f5a-15bc12758245 None admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:35:12.015171 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24d64c3-1913-4b87-9f5a-15bc12758245 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:35:12.016601 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24d64c3-1913-4b87-9f5a-15bc12758245 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:35:12.048209 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 580/1173] 10.222.0.58 () {60 vars in 1141 bytes} [Tue Jul 26 13:35:11 2022] POST /identity/v3/projects => generated 361 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:35:14.095185 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56deb827-50b0-41a1-bf75-b83f2767d6f4 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:35:14.095757 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56deb827-50b0-41a1-bf75-b83f2767d6f4 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:35:14.095757 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56deb827-50b0-41a1-bf75-b83f2767d6f4 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:35:14.097092 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 594/1174] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:35:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:35:14.102887 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7216bc-4380-4c77-87a2-339c79a18962 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:35:14.103296 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7216bc-4380-4c77-87a2-339c79a18962 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:35:14.103448 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7216bc-4380-4c77-87a2-339c79a18962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:35:14.122040 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-cf7216bc-4380-4c77-87a2-339c79a18962 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:35:14.123255 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-cf7216bc-4380-4c77-87a2-339c79a18962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:35:14.137180 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:35:14.193383 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 581/1175] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:35:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 13:35:14.218970 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56deb827-50b0-41a1-bf75-b83f2767d6f4 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:35:14.326062 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56deb827-50b0-41a1-bf75-b83f2767d6f4 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:35:14.360760 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c714714-6ce7-4a2f-910e-d780dd635281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:35:14.361406 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c714714-6ce7-4a2f-910e-d780dd635281 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:35:14.361717 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c714714-6ce7-4a2f-910e-d780dd635281 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:35:14.361993 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c714714-6ce7-4a2f-910e-d780dd635281 None admin] PATH_INFO: `/v3/domains/a35a8059601c455b9eca35e324840487` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:35:14.363177 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c714714-6ce7-4a2f-910e-d780dd635281 None admin] RBAC: Authorizing `identity:get_domain(domain_id=a35a8059601c455b9eca35e324840487)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:35:14.364601 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c714714-6ce7-4a2f-910e-d780dd635281 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:35:14.366250 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 595/1176] 10.222.0.58 () {56 vars in 1183 bytes} [Tue Jul 26 13:35:14 2022] GET /identity/v3/domains/a35a8059601c455b9eca35e324840487 => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:35:14.372121 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7216bc-4380-4c77-87a2-339c79a18962 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:35:14.472217 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7216bc-4380-4c77-87a2-339c79a18962 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:35:14.506332 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab725635-34a2-48e2-8427-851fd86c1a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:35:14.506946 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab725635-34a2-48e2-8427-851fd86c1a71 None admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:35:14.507229 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab725635-34a2-48e2-8427-851fd86c1a71 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:35:14.507505 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab725635-34a2-48e2-8427-851fd86c1a71 None admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:35:14.508188 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab725635-34a2-48e2-8427-851fd86c1a71 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:35:14.509644 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab725635-34a2-48e2-8427-851fd86c1a71 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:35:14.530846 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 582/1177] 10.222.0.58 () {60 vars in 1132 bytes} [Tue Jul 26 13:35:14 2022] POST /identity/v3/users => generated 311 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:35:16.592615 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09eac98-cc81-41c9-a3d5-70a80a1426f6 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:35:16.593165 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09eac98-cc81-41c9-a3d5-70a80a1426f6 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:35:16.593165 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09eac98-cc81-41c9-a3d5-70a80a1426f6 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:35:16.594571 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 596/1178] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:35:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:35:16.600219 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8958f4b8-42b2-45c4-a4c7-d72e79cdeb1a None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:35:16.600655 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8958f4b8-42b2-45c4-a4c7-d72e79cdeb1a None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:35:16.600757 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8958f4b8-42b2-45c4-a4c7-d72e79cdeb1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:35:16.607947 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:35:16.619339 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-8958f4b8-42b2-45c4-a4c7-d72e79cdeb1a None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:35:16.620538 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-8958f4b8-42b2-45c4-a4c7-d72e79cdeb1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:35:16.685058 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 583/1179] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:35:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 13:35:16.708226 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09eac98-cc81-41c9-a3d5-70a80a1426f6 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:35:16.735230 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:35:16.811212 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09eac98-cc81-41c9-a3d5-70a80a1426f6 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:35:16.844119 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4870e55-eed7-4598-955f-fc9d35e62328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:35:16.844691 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4870e55-eed7-4598-955f-fc9d35e62328 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:35:16.844974 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4870e55-eed7-4598-955f-fc9d35e62328 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:35:16.845249 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4870e55-eed7-4598-955f-fc9d35e62328 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:35:16.856429 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4870e55-eed7-4598-955f-fc9d35e62328 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:35:16.861357 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-f4870e55-eed7-4598-955f-fc9d35e62328 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 13:35:16.861357 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:35:16.861357 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:35:16.861357 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:35:16.861357 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:35:16.861357 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:35:16.861357 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:35:16.861357 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:35:16.861357 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:35:16.861357 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:35:16.861357 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:35:16.861357 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:35:16.861357 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:35:16.861357 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:35:16.861357 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:35:16.861357 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:35:16.862534 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:35:16.862534 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:35:16.862534 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:35:16.862534 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:35:16.862534 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:35:16.862534 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:35:16.862534 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:35:16.862534 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:35:16.862534 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:35:16.862534 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:35:16.862534 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:35:16.862534 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:35:16.862534 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:35:16.862534 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:35:16.862534 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:35:16.862534 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:35:16.863460 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:35:16.863460 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:35:16.863460 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:35:16.863460 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:35:16.863460 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:35:16.863460 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:35:16.863460 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:35:16.863460 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:35:16.863460 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:35:16.863460 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:35:16.863460 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 13:35:16.863460 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:35:16.863460 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 597/1180] 10.222.0.58 () {56 vars in 1096 bytes} [Tue Jul 26 13:35:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 13:35:16.868618 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8958f4b8-42b2-45c4-a4c7-d72e79cdeb1a None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:35:16.965720 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8958f4b8-42b2-45c4-a4c7-d72e79cdeb1a None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:35:16.997744 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d429ea23-d22e-4401-9ab8-97f5ba4c5c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:35:16.998350 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d429ea23-d22e-4401-9ab8-97f5ba4c5c76 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:35:16.998633 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d429ea23-d22e-4401-9ab8-97f5ba4c5c76 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:35:16.998908 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d429ea23-d22e-4401-9ab8-97f5ba4c5c76 None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:35:16.999616 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d429ea23-d22e-4401-9ab8-97f5ba4c5c76 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:35:16.999922 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d429ea23-d22e-4401-9ab8-97f5ba4c5c76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:35:17.001275 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d429ea23-d22e-4401-9ab8-97f5ba4c5c76 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:35:17.002640 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:35:17.010673 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 584/1181] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Jul 26 13:35:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:35:17.016741 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4870e55-eed7-4598-955f-fc9d35e62328 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:35:17.095968 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:35:17.113657 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4870e55-eed7-4598-955f-fc9d35e62328 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:35:17.146244 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65d8777-86d7-4d33-a82a-d0086a508d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:35:17.146880 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65d8777-86d7-4d33-a82a-d0086a508d80 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:35:17.147163 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65d8777-86d7-4d33-a82a-d0086a508d80 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:35:17.147443 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65d8777-86d7-4d33-a82a-d0086a508d80 None admin] PATH_INFO: `/v3/users/ca01d22b02db4911b9fc3f3be459b8e9` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:35:17.159979 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65d8777-86d7-4d33-a82a-d0086a508d80 None admin] RBAC: Authorizing `identity:get_user(user_id=ca01d22b02db4911b9fc3f3be459b8e9)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:35:17.161402 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65d8777-86d7-4d33-a82a-d0086a508d80 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:35:17.163095 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 598/1182] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Jul 26 13:35:17 2022] GET /identity/v3/users/ca01d22b02db4911b9fc3f3be459b8e9 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:35:17.168988 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d429ea23-d22e-4401-9ab8-97f5ba4c5c76 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:35:17.267836 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d429ea23-d22e-4401-9ab8-97f5ba4c5c76 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:35:17.301083 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a28f02-076b-481a-9cbe-1855d8ed60c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:35:17.301714 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a28f02-076b-481a-9cbe-1855d8ed60c9 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:35:17.301995 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a28f02-076b-481a-9cbe-1855d8ed60c9 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:35:17.302265 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a28f02-076b-481a-9cbe-1855d8ed60c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:35:17.360788 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:35:17.368330 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a28f02-076b-481a-9cbe-1855d8ed60c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:35:17.369784 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a28f02-076b-481a-9cbe-1855d8ed60c9 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:35:17.463423 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 585/1183] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Jul 26 13:35:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 13:35:17.469262 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65d8777-86d7-4d33-a82a-d0086a508d80 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:35:17.567079 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65d8777-86d7-4d33-a82a-d0086a508d80 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:35:17.601699 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe5b830-97df-43e2-a6f8-775b7752faa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:35:17.602398 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe5b830-97df-43e2-a6f8-775b7752faa3 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:35:17.602604 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe5b830-97df-43e2-a6f8-775b7752faa3 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:35:17.602877 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe5b830-97df-43e2-a6f8-775b7752faa3 None admin] PATH_INFO: `/v3/projects/d00737ca65c24ae8b2a2160db964044d` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:35:17.604035 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe5b830-97df-43e2-a6f8-775b7752faa3 None admin] RBAC: Authorizing `identity:get_project(project_id=d00737ca65c24ae8b2a2160db964044d)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:35:17.605450 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe5b830-97df-43e2-a6f8-775b7752faa3 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:35:17.607146 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 599/1184] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Jul 26 13:35:17 2022] GET /identity/v3/projects/d00737ca65c24ae8b2a2160db964044d => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:35:17.613989 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a28f02-076b-481a-9cbe-1855d8ed60c9 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:35:17.709045 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:35:17.716118 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a28f02-076b-481a-9cbe-1855d8ed60c9 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:35:17.752585 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad28adef-9560-4d90-a06e-063d8e7a852c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:35:17.753307 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad28adef-9560-4d90-a06e-063d8e7a852c None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:35:17.753652 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad28adef-9560-4d90-a06e-063d8e7a852c None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:35:17.753934 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad28adef-9560-4d90-a06e-063d8e7a852c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:35:17.754739 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad28adef-9560-4d90-a06e-063d8e7a852c None admin] RBAC: Adding query filter params (role.id=6e0201a9a6484f7d8b112b1eafff70bd, scope.project.id=d00737ca65c24ae8b2a2160db964044d, user.id=ca01d22b02db4911b9fc3f3be459b8e9) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:35:17.755052 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad28adef-9560-4d90-a06e-063d8e7a852c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:35:17.756492 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad28adef-9560-4d90-a06e-063d8e7a852c None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:35:17.762550 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 586/1185] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Jul 26 13:35:17 2022] GET /identity/v3/role_assignments?user.id=ca01d22b02db4911b9fc3f3be459b8e9&scope.project.id=d00737ca65c24ae8b2a2160db964044d&role.id=6e0201a9a6484f7d8b112b1eafff70bd => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:35:19.739789 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87dff33-4845-4efe-83b7-60081c5e1967 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:35:19.740337 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87dff33-4845-4efe-83b7-60081c5e1967 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:35:19.740337 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87dff33-4845-4efe-83b7-60081c5e1967 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:35:19.741707 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 600/1186] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:35:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:35:19.746966 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaf6aba-a1f6-457c-be43-77b53c38f1ee None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:35:19.747438 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaf6aba-a1f6-457c-be43-77b53c38f1ee None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:35:19.747541 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaf6aba-a1f6-457c-be43-77b53c38f1ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:35:19.765712 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-bbaf6aba-a1f6-457c-be43-77b53c38f1ee None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:35:19.766899 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-bbaf6aba-a1f6-457c-be43-77b53c38f1ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:35:19.833688 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 587/1187] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:35:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 13:35:19.856363 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87dff33-4845-4efe-83b7-60081c5e1967 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:35:19.867817 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:35:19.965509 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87dff33-4845-4efe-83b7-60081c5e1967 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:35:20.000609 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfac7df-8c6c-46d7-b9ad-d630ea1006c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:35:20.001167 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfac7df-8c6c-46d7-b9ad-d630ea1006c1 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:35:20.001448 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfac7df-8c6c-46d7-b9ad-d630ea1006c1 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:35:20.001841 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfac7df-8c6c-46d7-b9ad-d630ea1006c1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:35:20.012074 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfac7df-8c6c-46d7-b9ad-d630ea1006c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:35:20.016981 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-8dfac7df-8c6c-46d7-b9ad-d630ea1006c1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 13:35:20.016981 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:35:20.016981 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:35:20.016981 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:35:20.016981 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:35:20.016981 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:35:20.016981 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:35:20.016981 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:35:20.016981 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:35:20.016981 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:35:20.016981 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:35:20.016981 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:35:20.016981 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:35:20.016981 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:35:20.016981 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:35:20.016981 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:35:20.018080 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:35:20.018080 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:35:20.018080 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:35:20.018080 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:35:20.018080 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:35:20.018080 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:35:20.018080 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:35:20.018080 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:35:20.018080 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:35:20.018080 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:35:20.018080 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:35:20.018080 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:35:20.018080 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:35:20.018080 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:35:20.018080 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:35:20.018080 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:35:20.019153 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:35:20.019153 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:35:20.019153 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:35:20.019153 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:35:20.019153 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:35:20.019153 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:35:20.019153 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:35:20.019153 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:35:20.019153 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:35:20.019153 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:35:20.019153 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 13:35:20.019153 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:35:20.019153 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 601/1188] 10.222.0.58 () {56 vars in 1096 bytes} [Tue Jul 26 13:35:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:35:20.024358 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaf6aba-a1f6-457c-be43-77b53c38f1ee None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:35:20.118831 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:35:20.128584 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaf6aba-a1f6-457c-be43-77b53c38f1ee None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:35:20.162595 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bfa126-de6b-400d-a9b6-c89c7f6640ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:35:20.163210 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bfa126-de6b-400d-a9b6-c89c7f6640ea None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:35:20.163491 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bfa126-de6b-400d-a9b6-c89c7f6640ea None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:35:20.163765 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bfa126-de6b-400d-a9b6-c89c7f6640ea None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:35:20.164473 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bfa126-de6b-400d-a9b6-c89c7f6640ea None admin] RBAC: Adding query filter params (name=admin) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:35:20.164777 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bfa126-de6b-400d-a9b6-c89c7f6640ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:35:20.166167 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bfa126-de6b-400d-a9b6-c89c7f6640ea None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:35:20.174591 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 588/1189] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Jul 26 13:35:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:35:20.181106 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfac7df-8c6c-46d7-b9ad-d630ea1006c1 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:35:20.248139 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:35:20.281602 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfac7df-8c6c-46d7-b9ad-d630ea1006c1 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:35:20.315352 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dbf1e1-c015-4931-8710-5fca45741b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:35:20.316023 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dbf1e1-c015-4931-8710-5fca45741b49 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:35:20.316305 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dbf1e1-c015-4931-8710-5fca45741b49 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:35:20.316580 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dbf1e1-c015-4931-8710-5fca45741b49 None admin] PATH_INFO: `/v3/users/ca01d22b02db4911b9fc3f3be459b8e9` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:35:20.317761 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dbf1e1-c015-4931-8710-5fca45741b49 None admin] RBAC: Authorizing `identity:get_user(user_id=ca01d22b02db4911b9fc3f3be459b8e9)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:35:20.319186 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dbf1e1-c015-4931-8710-5fca45741b49 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:35:20.320801 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 602/1190] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Jul 26 13:35:20 2022] GET /identity/v3/users/ca01d22b02db4911b9fc3f3be459b8e9 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:35:20.326643 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bfa126-de6b-400d-a9b6-c89c7f6640ea None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:35:20.424337 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bfa126-de6b-400d-a9b6-c89c7f6640ea None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:35:20.456084 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3622e5-9fbf-4c17-81a4-eebad9532a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:35:20.457228 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3622e5-9fbf-4c17-81a4-eebad9532a9a None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:35:20.457528 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3622e5-9fbf-4c17-81a4-eebad9532a9a None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:35:20.457805 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3622e5-9fbf-4c17-81a4-eebad9532a9a None admin] PATH_INFO: `/v3/projects/d00737ca65c24ae8b2a2160db964044d` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:35:20.458960 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3622e5-9fbf-4c17-81a4-eebad9532a9a None admin] RBAC: Authorizing `identity:get_project(project_id=d00737ca65c24ae8b2a2160db964044d)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:35:20.460375 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3622e5-9fbf-4c17-81a4-eebad9532a9a None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:35:20.462056 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 589/1191] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Jul 26 13:35:20 2022] GET /identity/v3/projects/d00737ca65c24ae8b2a2160db964044d => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:35:20.468038 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dbf1e1-c015-4931-8710-5fca45741b49 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:35:20.565023 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dbf1e1-c015-4931-8710-5fca45741b49 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:35:20.599202 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e9e805-6ff4-40e5-9e2f-98de2a6e2f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:35:20.599871 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e9e805-6ff4-40e5-9e2f-98de2a6e2f91 None admin] REQUEST_METHOD: `PUT` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:35:20.600150 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e9e805-6ff4-40e5-9e2f-98de2a6e2f91 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:35:20.600427 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e9e805-6ff4-40e5-9e2f-98de2a6e2f91 None admin] PATH_INFO: `/v3/projects/d00737ca65c24ae8b2a2160db964044d/users/ca01d22b02db4911b9fc3f3be459b8e9/roles/6e0201a9a6484f7d8b112b1eafff70bd` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:35:20.602353 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e9e805-6ff4-40e5-9e2f-98de2a6e2f91 None admin] RBAC: Authorizing `identity:create_grant(project_id=d00737ca65c24ae8b2a2160db964044d, user_id=ca01d22b02db4911b9fc3f3be459b8e9, role_id=6e0201a9a6484f7d8b112b1eafff70bd)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:35:20.603767 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e9e805-6ff4-40e5-9e2f-98de2a6e2f91 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:35:20.606735 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:35:20.618223 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 603/1192] 10.222.0.58 () {58 vars in 1439 bytes} [Tue Jul 26 13:35:20 2022] PUT /identity/v3/projects/d00737ca65c24ae8b2a2160db964044d/users/ca01d22b02db4911b9fc3f3be459b8e9/roles/6e0201a9a6484f7d8b112b1eafff70bd => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:35:22.587387 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad2816e-c77a-42b8-9cbb-5063c7be436c None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:35:22.587973 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad2816e-c77a-42b8-9cbb-5063c7be436c None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:35:22.588094 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad2816e-c77a-42b8-9cbb-5063c7be436c None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:35:22.589518 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 590/1193] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:35:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:35:22.594992 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c7494b-4012-4bdd-9747-a667ff254f06 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:35:22.595475 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c7494b-4012-4bdd-9747-a667ff254f06 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:35:22.595565 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c7494b-4012-4bdd-9747-a667ff254f06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:35:22.614591 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-19c7494b-4012-4bdd-9747-a667ff254f06 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:35:22.615803 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-19c7494b-4012-4bdd-9747-a667ff254f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:35:22.682497 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 604/1194] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:35:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 13:35:22.705884 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad2816e-c77a-42b8-9cbb-5063c7be436c None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:35:22.728006 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:35:22.811537 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad2816e-c77a-42b8-9cbb-5063c7be436c None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:35:22.844522 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579e4c40-bab1-4cb4-b961-289838c95b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:35:22.845082 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579e4c40-bab1-4cb4-b961-289838c95b2d None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:35:22.845367 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579e4c40-bab1-4cb4-b961-289838c95b2d None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:35:22.845671 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579e4c40-bab1-4cb4-b961-289838c95b2d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:35:22.856929 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579e4c40-bab1-4cb4-b961-289838c95b2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:35:22.861845 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-579e4c40-bab1-4cb4-b961-289838c95b2d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 13:35:22.861845 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:35:22.861845 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:35:22.861845 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:35:22.861845 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:35:22.861845 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:35:22.861845 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:35:22.861845 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:35:22.861845 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:35:22.861845 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:35:22.861845 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:35:22.861845 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:35:22.861845 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:35:22.861845 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:35:22.861845 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:35:22.861845 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:35:22.863531 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:35:22.863531 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:35:22.863531 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:35:22.863531 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:35:22.863531 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:35:22.863531 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:35:22.863531 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:35:22.863531 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:35:22.863531 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:35:22.863531 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:35:22.863531 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:35:22.863531 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:35:22.863531 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:35:22.863531 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:35:22.863531 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:35:22.863531 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:35:22.864568 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:35:22.864568 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:35:22.864568 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:35:22.864568 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:35:22.864568 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:35:22.864568 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:35:22.864568 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:35:22.864568 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:35:22.864568 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:35:22.864568 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:35:22.864568 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 13:35:22.864568 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:35:22.864568 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 591/1195] 10.222.0.58 () {56 vars in 1096 bytes} [Tue Jul 26 13:35:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:35:22.869855 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c7494b-4012-4bdd-9747-a667ff254f06 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:35:22.968133 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c7494b-4012-4bdd-9747-a667ff254f06 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:35:22.993537 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:35:23.002698 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bdb218-4c97-4fb2-9bbf-39c833741681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:35:23.003345 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bdb218-4c97-4fb2-9bbf-39c833741681 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:35:23.003632 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bdb218-4c97-4fb2-9bbf-39c833741681 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:35:23.003914 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bdb218-4c97-4fb2-9bbf-39c833741681 None admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:35:23.004634 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bdb218-4c97-4fb2-9bbf-39c833741681 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:35:23.004940 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bdb218-4c97-4fb2-9bbf-39c833741681 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:35:23.006387 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bdb218-4c97-4fb2-9bbf-39c833741681 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:35:23.014618 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 605/1196] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Jul 26 13:35:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:35:23.020410 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579e4c40-bab1-4cb4-b961-289838c95b2d None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:35:23.100434 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:35:23.120315 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579e4c40-bab1-4cb4-b961-289838c95b2d None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:35:23.152635 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc2c64b-ccb4-4497-ae40-c253b39cc869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:35:23.153272 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc2c64b-ccb4-4497-ae40-c253b39cc869 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:35:23.153584 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc2c64b-ccb4-4497-ae40-c253b39cc869 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:35:23.153864 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc2c64b-ccb4-4497-ae40-c253b39cc869 None admin] PATH_INFO: `/v3/users/ca01d22b02db4911b9fc3f3be459b8e9` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:35:23.155021 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc2c64b-ccb4-4497-ae40-c253b39cc869 None admin] RBAC: Authorizing `identity:get_user(user_id=ca01d22b02db4911b9fc3f3be459b8e9)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:35:23.156458 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc2c64b-ccb4-4497-ae40-c253b39cc869 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:35:23.158270 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 592/1197] 10.222.0.58 () {56 vars in 1177 bytes} [Tue Jul 26 13:35:23 2022] GET /identity/v3/users/ca01d22b02db4911b9fc3f3be459b8e9 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:35:23.164371 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bdb218-4c97-4fb2-9bbf-39c833741681 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:35:23.265066 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bdb218-4c97-4fb2-9bbf-39c833741681 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:35:23.298513 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61cf1bf-623c-4b01-815f-22bc0c5a9203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:35:23.299128 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61cf1bf-623c-4b01-815f-22bc0c5a9203 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:35:23.299414 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61cf1bf-623c-4b01-815f-22bc0c5a9203 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:35:23.299688 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61cf1bf-623c-4b01-815f-22bc0c5a9203 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:35:23.354242 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:35:23.368263 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61cf1bf-623c-4b01-815f-22bc0c5a9203 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:35:23.369753 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61cf1bf-623c-4b01-815f-22bc0c5a9203 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:35:23.467476 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 606/1198] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Jul 26 13:35:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 13:35:23.474146 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc2c64b-ccb4-4497-ae40-c253b39cc869 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:35:23.573530 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc2c64b-ccb4-4497-ae40-c253b39cc869 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:35:23.607064 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89a1a95-5814-4860-a6a0-595159f7bb57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:35:23.607700 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89a1a95-5814-4860-a6a0-595159f7bb57 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:35:23.607888 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89a1a95-5814-4860-a6a0-595159f7bb57 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:35:23.608166 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89a1a95-5814-4860-a6a0-595159f7bb57 None admin] PATH_INFO: `/v3/projects/d00737ca65c24ae8b2a2160db964044d` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:35:23.609325 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89a1a95-5814-4860-a6a0-595159f7bb57 None admin] RBAC: Authorizing `identity:get_project(project_id=d00737ca65c24ae8b2a2160db964044d)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:35:23.610795 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89a1a95-5814-4860-a6a0-595159f7bb57 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:35:23.612481 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 593/1199] 10.222.0.58 () {56 vars in 1186 bytes} [Tue Jul 26 13:35:23 2022] GET /identity/v3/projects/d00737ca65c24ae8b2a2160db964044d => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:35:23.618677 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61cf1bf-623c-4b01-815f-22bc0c5a9203 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:35:23.706004 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:35:23.718438 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61cf1bf-623c-4b01-815f-22bc0c5a9203 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:35:23.756347 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301e487f-ffe1-4203-8aa5-e0d1463542dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:35:23.756965 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301e487f-ffe1-4203-8aa5-e0d1463542dc None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:35:23.757246 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301e487f-ffe1-4203-8aa5-e0d1463542dc None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:35:23.757548 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301e487f-ffe1-4203-8aa5-e0d1463542dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:35:23.758338 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301e487f-ffe1-4203-8aa5-e0d1463542dc None admin] RBAC: Adding query filter params (role.id=6e0201a9a6484f7d8b112b1eafff70bd, scope.project.id=d00737ca65c24ae8b2a2160db964044d, user.id=ca01d22b02db4911b9fc3f3be459b8e9) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:35:23.758650 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301e487f-ffe1-4203-8aa5-e0d1463542dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:35:23.760059 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301e487f-ffe1-4203-8aa5-e0d1463542dc None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:35:23.766038 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 607/1200] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Jul 26 13:35:23 2022] GET /identity/v3/role_assignments?user.id=ca01d22b02db4911b9fc3f3be459b8e9&scope.project.id=d00737ca65c24ae8b2a2160db964044d&role.id=6e0201a9a6484f7d8b112b1eafff70bd => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:35:33.602688 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=92 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:35:33.763274 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=24 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:36:18.216872 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c0b698-7fe0-44e1-8631-c38504aa2a16 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:18.217504 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c0b698-7fe0-44e1-8631-c38504aa2a16 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:18.217504 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c0b698-7fe0-44e1-8631-c38504aa2a16 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:18.218974 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 594/1201] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:36:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:36:18.224944 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9644316d-b69e-4b1a-80f5-d5bf95b73e8d None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:18.225438 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9644316d-b69e-4b1a-80f5-d5bf95b73e8d None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:18.225748 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9644316d-b69e-4b1a-80f5-d5bf95b73e8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:18.244962 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-9644316d-b69e-4b1a-80f5-d5bf95b73e8d None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:36:18.246210 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-9644316d-b69e-4b1a-80f5-d5bf95b73e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:36:18.333279 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 608/1202] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:36:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 13:36:18.357582 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c0b698-7fe0-44e1-8631-c38504aa2a16 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:36:18.491643 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c0b698-7fe0-44e1-8631-c38504aa2a16 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:36:18.530783 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88247047-4287-4a4c-baf2-9596cbcdf1fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:36:18.531405 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88247047-4287-4a4c-baf2-9596cbcdf1fb None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:18.531735 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88247047-4287-4a4c-baf2-9596cbcdf1fb None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:18.532041 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88247047-4287-4a4c-baf2-9596cbcdf1fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:18.544321 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88247047-4287-4a4c-baf2-9596cbcdf1fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:36:18.545813 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88247047-4287-4a4c-baf2-9596cbcdf1fb None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:36:18.561173 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-88247047-4287-4a4c-baf2-9596cbcdf1fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:36:18.561173 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:36:18.561173 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:36:18.561173 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:36:18.561173 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:36:18.561173 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:36:18.561173 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:36:18.561173 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:36:18.561173 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:36:18.561173 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:36:18.561173 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:36:18.561173 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:36:18.561173 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 13:36:18.561173 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 13:36:18.561173 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 13:36:18.562479 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 13:36:18.562479 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:36:18.562479 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:36:18.562479 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:36:18.562479 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:36:18.562479 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:36:18.562479 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:36:18.562479 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:36:18.562479 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:36:18.562479 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:36:18.562479 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:36:18.562479 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:36:18.562479 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:36:18.562479 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:36:18.562479 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:36:18.562479 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:36:18.562479 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:36:18.564072 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 13:36:18.564072 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 13:36:18.564072 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:36:18.564072 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:36:18.564072 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 595/1203] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:36:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:36:18.569065 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9644316d-b69e-4b1a-80f5-d5bf95b73e8d None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:36:18.704266 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9644316d-b69e-4b1a-80f5-d5bf95b73e8d None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:36:18.738717 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:36:18.746069 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e82a5b-209f-4b7c-bc87-481c43222772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:36:18.746784 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e82a5b-209f-4b7c-bc87-481c43222772 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:18.747073 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e82a5b-209f-4b7c-bc87-481c43222772 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:18.747358 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e82a5b-209f-4b7c-bc87-481c43222772 None admin] PATH_INFO: `/v3/domains` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:18.748191 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e82a5b-209f-4b7c-bc87-481c43222772 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:36:18.748502 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e82a5b-209f-4b7c-bc87-481c43222772 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:36:18.749991 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e82a5b-209f-4b7c-bc87-481c43222772 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:36:18.768110 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 609/1204] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Jul 26 13:36:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:36:18.774170 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88247047-4287-4a4c-baf2-9596cbcdf1fb None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:36:18.855917 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:36:18.923055 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88247047-4287-4a4c-baf2-9596cbcdf1fb None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:36:18.967391 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f91818-159c-4f1e-baff-4c27f8f2d52b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:36:18.967917 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f91818-159c-4f1e-baff-4c27f8f2d52b None admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:18.968199 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f91818-159c-4f1e-baff-4c27f8f2d52b None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:18.968468 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f91818-159c-4f1e-baff-4c27f8f2d52b None admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:18.969741 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f91818-159c-4f1e-baff-4c27f8f2d52b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:36:18.971174 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f91818-159c-4f1e-baff-4c27f8f2d52b None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:36:19.003534 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 596/1205] 10.222.0.58 () {60 vars in 1132 bytes} [Tue Jul 26 13:36:18 2022] POST /identity/v3/users => generated 251 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:36:21.100816 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ae0090-0497-4488-b97b-6bedff43074e None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:21.101379 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ae0090-0497-4488-b97b-6bedff43074e None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:21.101379 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ae0090-0497-4488-b97b-6bedff43074e None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:21.102752 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 610/1206] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:36:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:36:21.108278 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9784f205-e0f1-4cd6-90c9-1d0f178d6c9b None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:21.108729 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9784f205-e0f1-4cd6-90c9-1d0f178d6c9b None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:21.108822 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9784f205-e0f1-4cd6-90c9-1d0f178d6c9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:21.131289 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-9784f205-e0f1-4cd6-90c9-1d0f178d6c9b None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:36:21.132769 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-9784f205-e0f1-4cd6-90c9-1d0f178d6c9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:36:21.219006 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 597/1207] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:36:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 13:36:21.244934 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ae0090-0497-4488-b97b-6bedff43074e None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:36:21.365882 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ae0090-0497-4488-b97b-6bedff43074e None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:36:21.403996 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d03426-6b94-4d2a-914b-46c4908581f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:36:21.404646 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d03426-6b94-4d2a-914b-46c4908581f5 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:21.404944 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d03426-6b94-4d2a-914b-46c4908581f5 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:21.405207 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d03426-6b94-4d2a-914b-46c4908581f5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:21.417408 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d03426-6b94-4d2a-914b-46c4908581f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:36:21.422883 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-f6d03426-6b94-4d2a-914b-46c4908581f5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 13:36:21.422883 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:36:21.422883 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:36:21.422883 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:36:21.422883 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:36:21.422883 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:36:21.422883 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:36:21.422883 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:36:21.422883 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:36:21.422883 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:36:21.422883 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:36:21.422883 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:36:21.422883 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:36:21.422883 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:36:21.422883 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:36:21.422883 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:36:21.424019 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:36:21.424019 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:36:21.424019 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:36:21.424019 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:36:21.424019 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:36:21.424019 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:36:21.424019 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:36:21.424019 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:36:21.424019 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:36:21.424019 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:36:21.424019 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:36:21.424019 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:36:21.424019 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:36:21.424019 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:36:21.424019 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:36:21.424019 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:36:21.425371 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:36:21.425371 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:36:21.425371 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:36:21.425371 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:36:21.425371 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:36:21.425371 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:36:21.425371 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:36:21.425371 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:36:21.425371 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:36:21.425371 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:36:21.425371 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 13:36:21.425371 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:36:21.425371 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 611/1208] 10.222.0.58 () {56 vars in 1102 bytes} [Tue Jul 26 13:36:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 13:36:21.430744 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9784f205-e0f1-4cd6-90c9-1d0f178d6c9b None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:36:21.475662 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:36:21.542224 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9784f205-e0f1-4cd6-90c9-1d0f178d6c9b None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:36:21.580096 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8872e7-513d-43ed-9ba8-53b83dfa20e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:36:21.580808 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8872e7-513d-43ed-9ba8-53b83dfa20e3 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:21.581162 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8872e7-513d-43ed-9ba8-53b83dfa20e3 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:21.581530 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8872e7-513d-43ed-9ba8-53b83dfa20e3 None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:21.582319 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8872e7-513d-43ed-9ba8-53b83dfa20e3 None admin] RBAC: Adding query filter params (name=service) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:36:21.584341 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8872e7-513d-43ed-9ba8-53b83dfa20e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:36:21.584341 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8872e7-513d-43ed-9ba8-53b83dfa20e3 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:36:21.592669 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 598/1209] 10.222.0.58 () {56 vars in 1103 bytes} [Tue Jul 26 13:36:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:36:21.598989 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d03426-6b94-4d2a-914b-46c4908581f5 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:36:21.652360 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:36:21.709961 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d03426-6b94-4d2a-914b-46c4908581f5 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:36:21.752112 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb74e0ba-9f77-46db-b84e-ec5d944f495e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:36:21.753135 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb74e0ba-9f77-46db-b84e-ec5d944f495e None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:21.753498 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb74e0ba-9f77-46db-b84e-ec5d944f495e None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:21.753843 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb74e0ba-9f77-46db-b84e-ec5d944f495e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:21.768293 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb74e0ba-9f77-46db-b84e-ec5d944f495e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:36:21.770114 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb74e0ba-9f77-46db-b84e-ec5d944f495e None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:36:21.788328 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-eb74e0ba-9f77-46db-b84e-ec5d944f495e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:36:21.788328 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:36:21.788328 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:36:21.788328 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:36:21.788328 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:36:21.788328 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:36:21.788328 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:36:21.788328 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:36:21.788328 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:36:21.788328 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:36:21.788328 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:36:21.788328 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:36:21.788328 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 13:36:21.788328 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 13:36:21.788328 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 13:36:21.789525 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 13:36:21.789525 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:36:21.789525 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:36:21.789525 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:36:21.789525 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:36:21.789525 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:36:21.789525 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:36:21.789525 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:36:21.789525 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:36:21.789525 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:36:21.789525 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:36:21.789525 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:36:21.789525 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:36:21.789525 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:36:21.789525 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:36:21.789525 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:36:21.789525 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:36:21.794612 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 13:36:21.794612 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 13:36:21.794612 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:36:21.794612 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:36:21.794612 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 612/1210] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:36:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:36:21.797721 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8872e7-513d-43ed-9ba8-53b83dfa20e3 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:36:21.905460 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:36:21.910934 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8872e7-513d-43ed-9ba8-53b83dfa20e3 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:36:21.949358 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2573c394-5610-481d-b591-1502b16aba4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:36:21.950096 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2573c394-5610-481d-b591-1502b16aba4d None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:21.950451 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2573c394-5610-481d-b591-1502b16aba4d None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:21.950805 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2573c394-5610-481d-b591-1502b16aba4d None admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:21.951561 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2573c394-5610-481d-b591-1502b16aba4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:36:21.951975 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2573c394-5610-481d-b591-1502b16aba4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:36:21.953937 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2573c394-5610-481d-b591-1502b16aba4d None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:36:21.966972 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 599/1211] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Jul 26 13:36:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:36:21.973014 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb74e0ba-9f77-46db-b84e-ec5d944f495e None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:36:22.083211 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb74e0ba-9f77-46db-b84e-ec5d944f495e None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:36:22.084911 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:36:22.123025 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f527a5-c6b5-4b7a-bf91-a2e77783ce2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:36:22.123702 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f527a5-c6b5-4b7a-bf91-a2e77783ce2c None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:22.125859 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f527a5-c6b5-4b7a-bf91-a2e77783ce2c None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:22.125859 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f527a5-c6b5-4b7a-bf91-a2e77783ce2c None admin] PATH_INFO: `/v3/users/placement` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:22.133732 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f527a5-c6b5-4b7a-bf91-a2e77783ce2c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:36:22.135219 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f527a5-c6b5-4b7a-bf91-a2e77783ce2c None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:36:22.149561 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-89f527a5-c6b5-4b7a-bf91-a2e77783ce2c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 26 13:36:22.149561 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:36:22.149561 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:36:22.149561 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:36:22.149561 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:36:22.149561 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:36:22.149561 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:36:22.149561 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:36:22.149561 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:36:22.149561 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:36:22.149561 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:36:22.149561 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:36:22.149561 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 13:36:22.149561 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 13:36:22.149561 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 13:36:22.149561 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 13:36:22.150631 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:36:22.150631 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:36:22.150631 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 13:36:22.150631 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 13:36:22.150631 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 13:36:22.150631 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 13:36:22.150631 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:36:22.150631 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:36:22.150631 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:36:22.150631 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:36:22.150631 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:36:22.150631 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:36:22.150631 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:36:22.150631 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:36:22.150631 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:36:22.150631 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:36:22.150631 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:36:22.150631 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:36:22.151598 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:36:22.151598 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:36:22.151598 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 13:36:22.151598 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 13:36:22.151598 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 13:36:22.151598 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 13:36:22.151598 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 13:36:22.151598 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 13:36:22.151598 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 26 13:36:22.151598 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:36:22.151598 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 613/1212] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:36:21 2022] GET /identity/v3/users/placement => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:36:22.157199 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2573c394-5610-481d-b591-1502b16aba4d None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:36:22.277487 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2573c394-5610-481d-b591-1502b16aba4d None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:36:22.323700 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdcbe08-c192-41e4-b4ba-d245b72aa765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:36:22.324543 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdcbe08-c192-41e4-b4ba-d245b72aa765 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:22.324861 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdcbe08-c192-41e4-b4ba-d245b72aa765 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:22.325110 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdcbe08-c192-41e4-b4ba-d245b72aa765 None admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:22.325957 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdcbe08-c192-41e4-b4ba-d245b72aa765 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:36:22.326300 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdcbe08-c192-41e4-b4ba-d245b72aa765 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:36:22.327707 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdcbe08-c192-41e4-b4ba-d245b72aa765 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:36:22.347733 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 600/1213] 10.222.0.58 () {56 vars in 1143 bytes} [Tue Jul 26 13:36:22 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:36:22.353205 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f527a5-c6b5-4b7a-bf91-a2e77783ce2c None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:36:22.475144 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f527a5-c6b5-4b7a-bf91-a2e77783ce2c None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:36:22.524936 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbbcee7-c9bf-4977-a24c-ee8f16772552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:36:22.525784 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbbcee7-c9bf-4977-a24c-ee8f16772552 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:22.526063 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbbcee7-c9bf-4977-a24c-ee8f16772552 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:22.526473 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbbcee7-c9bf-4977-a24c-ee8f16772552 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:22.559599 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:36:22.608535 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbbcee7-c9bf-4977-a24c-ee8f16772552 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:36:22.610592 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbbcee7-c9bf-4977-a24c-ee8f16772552 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:36:22.729460 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 614/1214] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Jul 26 13:36:22 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 13:36:22.730266 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdcbe08-c192-41e4-b4ba-d245b72aa765 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:36:22.753950 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:36:22.860488 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdcbe08-c192-41e4-b4ba-d245b72aa765 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:36:22.900142 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b66398-5852-45af-8376-bf4a2e505ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:36:22.900968 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b66398-5852-45af-8376-bf4a2e505ee8 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:22.901344 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b66398-5852-45af-8376-bf4a2e505ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:22.901746 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b66398-5852-45af-8376-bf4a2e505ee8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:22.917230 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b66398-5852-45af-8376-bf4a2e505ee8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:36:22.919101 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b66398-5852-45af-8376-bf4a2e505ee8 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:36:22.936991 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-79b66398-5852-45af-8376-bf4a2e505ee8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:36:22.936991 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:36:22.936991 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:36:22.936991 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:36:22.936991 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:36:22.936991 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:36:22.936991 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:36:22.936991 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:36:22.936991 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:36:22.936991 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:36:22.936991 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:36:22.936991 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:36:22.936991 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 13:36:22.936991 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 13:36:22.936991 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 13:36:22.938241 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 13:36:22.938241 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:36:22.938241 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:36:22.938241 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:36:22.938241 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:36:22.938241 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:36:22.938241 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:36:22.938241 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:36:22.938241 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:36:22.938241 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:36:22.938241 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:36:22.938241 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:36:22.938241 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:36:22.938241 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:36:22.938241 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:36:22.938241 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:36:22.938241 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:36:22.939584 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 13:36:22.939584 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 13:36:22.939584 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:36:22.939584 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:36:22.939584 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 601/1215] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:36:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:36:22.944499 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbbcee7-c9bf-4977-a24c-ee8f16772552 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:36:23.015237 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:36:23.057497 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbbcee7-c9bf-4977-a24c-ee8f16772552 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:36:23.094434 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59d9c4f-347a-458a-b4db-e2d8076e0518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:36:23.095054 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59d9c4f-347a-458a-b4db-e2d8076e0518 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:23.095330 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59d9c4f-347a-458a-b4db-e2d8076e0518 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:23.095597 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59d9c4f-347a-458a-b4db-e2d8076e0518 None admin] PATH_INFO: `/v3/domains` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:23.096282 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59d9c4f-347a-458a-b4db-e2d8076e0518 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:36:23.096584 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59d9c4f-347a-458a-b4db-e2d8076e0518 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:36:23.097966 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59d9c4f-347a-458a-b4db-e2d8076e0518 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:36:23.110456 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 615/1216] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Jul 26 13:36:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:36:23.116369 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b66398-5852-45af-8376-bf4a2e505ee8 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:36:23.220179 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:36:23.268769 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b66398-5852-45af-8376-bf4a2e505ee8 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:36:23.314825 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949e25ba-43f1-428a-997f-a3239cc598f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:36:23.315490 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949e25ba-43f1-428a-997f-a3239cc598f0 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:23.315804 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949e25ba-43f1-428a-997f-a3239cc598f0 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:23.316026 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949e25ba-43f1-428a-997f-a3239cc598f0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:23.325132 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949e25ba-43f1-428a-997f-a3239cc598f0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:36:23.327239 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949e25ba-43f1-428a-997f-a3239cc598f0 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:36:23.347197 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-949e25ba-43f1-428a-997f-a3239cc598f0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 13:36:23.347197 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:36:23.347197 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:36:23.347197 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:36:23.347197 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:36:23.347197 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:36:23.347197 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:36:23.347197 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:36:23.347197 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:36:23.347197 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:36:23.347197 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:36:23.347197 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:36:23.347197 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 13:36:23.347197 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 13:36:23.347197 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 13:36:23.348568 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 13:36:23.348568 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:36:23.348568 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:36:23.348568 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:36:23.348568 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:36:23.348568 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:36:23.348568 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:36:23.348568 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:36:23.348568 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:36:23.348568 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:36:23.348568 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:36:23.348568 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:36:23.348568 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:36:23.348568 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:36:23.348568 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:36:23.348568 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:36:23.348568 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:36:23.350570 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 13:36:23.350570 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 13:36:23.350570 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 13:36:23.350570 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 13:36:23.350570 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 13:36:23.350570 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 13:36:23.350570 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 13:36:23.350570 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:36:23.350570 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 602/1217] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Jul 26 13:36:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:36:23.354663 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59d9c4f-347a-458a-b4db-e2d8076e0518 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:36:23.471282 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59d9c4f-347a-458a-b4db-e2d8076e0518 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:36:23.493997 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:36:23.508549 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fafdd60-2768-43a6-90ed-8e8e228878b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:36:23.509793 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fafdd60-2768-43a6-90ed-8e8e228878b7 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:23.509793 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fafdd60-2768-43a6-90ed-8e8e228878b7 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:23.510077 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fafdd60-2768-43a6-90ed-8e8e228878b7 None admin] PATH_INFO: `/v3/projects` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:23.511019 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fafdd60-2768-43a6-90ed-8e8e228878b7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:36:23.511366 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fafdd60-2768-43a6-90ed-8e8e228878b7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:36:23.512899 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fafdd60-2768-43a6-90ed-8e8e228878b7 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:36:23.532992 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 616/1218] 10.222.0.58 () {56 vars in 1148 bytes} [Tue Jul 26 13:36:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:36:23.534651 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949e25ba-43f1-428a-997f-a3239cc598f0 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:36:23.650147 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949e25ba-43f1-428a-997f-a3239cc598f0 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:36:23.689708 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:36:23.697554 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec13e588-dc06-4ee9-ad04-d0f4cf6909a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:36:23.698389 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec13e588-dc06-4ee9-ad04-d0f4cf6909a1 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:23.698705 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec13e588-dc06-4ee9-ad04-d0f4cf6909a1 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:23.699047 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec13e588-dc06-4ee9-ad04-d0f4cf6909a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:23.700137 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec13e588-dc06-4ee9-ad04-d0f4cf6909a1 None admin] RBAC: Adding query filter params (role.id=17e0f3c73a8a4a54b3e7b78878dfbf09, scope.project.id=1e8bb4385c204210a1bc77b1f0d654f3, user.id=835cbc13756b41b598142df3e35f5492) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:36:23.700484 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec13e588-dc06-4ee9-ad04-d0f4cf6909a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:36:23.702195 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec13e588-dc06-4ee9-ad04-d0f4cf6909a1 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:36:23.718206 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 603/1219] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Jul 26 13:36:23 2022] GET /identity/v3/role_assignments?user.id=835cbc13756b41b598142df3e35f5492&scope.project.id=1e8bb4385c204210a1bc77b1f0d654f3&role.id=17e0f3c73a8a4a54b3e7b78878dfbf09 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:36:25.883909 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378c8c55-0293-4ee4-8268-450205d12033 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:25.884475 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378c8c55-0293-4ee4-8268-450205d12033 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:25.884475 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378c8c55-0293-4ee4-8268-450205d12033 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:25.885908 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 617/1220] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:36:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:36:25.891568 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3ecb9b-bf26-43f5-9148-1a2a015d5820 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:25.891982 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3ecb9b-bf26-43f5-9148-1a2a015d5820 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:25.892236 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3ecb9b-bf26-43f5-9148-1a2a015d5820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:25.911531 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-3d3ecb9b-bf26-43f5-9148-1a2a015d5820 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:36:25.912731 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-3d3ecb9b-bf26-43f5-9148-1a2a015d5820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:36:25.984702 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 604/1221] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:36:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 13:36:26.007827 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378c8c55-0293-4ee4-8268-450205d12033 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:36:26.128607 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378c8c55-0293-4ee4-8268-450205d12033 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:36:26.164732 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b1b8d1-4e3d-4731-babc-5868d7202d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:36:26.165337 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b1b8d1-4e3d-4731-babc-5868d7202d85 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:26.165621 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b1b8d1-4e3d-4731-babc-5868d7202d85 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:26.165901 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b1b8d1-4e3d-4731-babc-5868d7202d85 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:26.177377 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b1b8d1-4e3d-4731-babc-5868d7202d85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:36:26.182395 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-c6b1b8d1-4e3d-4731-babc-5868d7202d85 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 13:36:26.182395 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:36:26.182395 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:36:26.182395 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:36:26.182395 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:36:26.182395 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:36:26.182395 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:36:26.182395 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:36:26.182395 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:36:26.182395 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:36:26.182395 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:36:26.182395 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:36:26.182395 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:36:26.182395 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:36:26.182395 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:36:26.182395 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:36:26.183821 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:36:26.183821 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:36:26.183821 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:36:26.183821 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:36:26.183821 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:36:26.183821 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:36:26.183821 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:36:26.183821 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:36:26.183821 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:36:26.183821 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:36:26.183821 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:36:26.183821 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:36:26.183821 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:36:26.183821 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:36:26.183821 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:36:26.183821 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:36:26.184670 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:36:26.184670 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:36:26.184670 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:36:26.184670 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:36:26.184670 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:36:26.184670 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:36:26.184670 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:36:26.184670 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:36:26.184670 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:36:26.184670 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:36:26.184670 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 13:36:26.184670 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:36:26.184670 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 618/1222] 10.222.0.58 () {56 vars in 1102 bytes} [Tue Jul 26 13:36:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 13:36:26.190232 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3ecb9b-bf26-43f5-9148-1a2a015d5820 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:36:26.293188 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3ecb9b-bf26-43f5-9148-1a2a015d5820 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:36:26.317581 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:36:26.328979 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c5c954-ab7b-4243-9f54-313bdecf7b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:36:26.329736 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c5c954-ab7b-4243-9f54-313bdecf7b29 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:26.330042 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c5c954-ab7b-4243-9f54-313bdecf7b29 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:26.330352 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c5c954-ab7b-4243-9f54-313bdecf7b29 None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:26.331132 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c5c954-ab7b-4243-9f54-313bdecf7b29 None admin] RBAC: Adding query filter params (name=service) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:36:26.331486 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c5c954-ab7b-4243-9f54-313bdecf7b29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:36:26.332921 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c5c954-ab7b-4243-9f54-313bdecf7b29 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:36:26.341719 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 605/1223] 10.222.0.58 () {56 vars in 1103 bytes} [Tue Jul 26 13:36:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:36:26.347916 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b1b8d1-4e3d-4731-babc-5868d7202d85 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:36:26.397384 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:36:26.462737 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b1b8d1-4e3d-4731-babc-5868d7202d85 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:36:26.501250 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fb8104-b6ff-47c9-bca6-4ef37a5dd200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:36:26.501926 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fb8104-b6ff-47c9-bca6-4ef37a5dd200 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:26.502211 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fb8104-b6ff-47c9-bca6-4ef37a5dd200 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:26.502488 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fb8104-b6ff-47c9-bca6-4ef37a5dd200 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:26.512485 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fb8104-b6ff-47c9-bca6-4ef37a5dd200 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:36:26.513951 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fb8104-b6ff-47c9-bca6-4ef37a5dd200 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:36:26.529263 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-43fb8104-b6ff-47c9-bca6-4ef37a5dd200 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:36:26.529263 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:36:26.529263 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:36:26.529263 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:36:26.529263 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:36:26.529263 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:36:26.529263 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:36:26.529263 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:36:26.529263 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:36:26.529263 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:36:26.529263 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:36:26.529263 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:36:26.529263 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 13:36:26.529263 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 13:36:26.529263 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 13:36:26.530378 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 13:36:26.530378 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:36:26.530378 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:36:26.530378 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:36:26.530378 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:36:26.530378 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:36:26.530378 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:36:26.530378 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:36:26.530378 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:36:26.530378 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:36:26.530378 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:36:26.530378 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:36:26.530378 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:36:26.530378 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:36:26.530378 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:36:26.530378 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:36:26.530378 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:36:26.532000 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 13:36:26.532000 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 13:36:26.532000 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:36:26.532000 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:36:26.532000 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 619/1224] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:36:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:36:26.536798 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c5c954-ab7b-4243-9f54-313bdecf7b29 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:36:26.642014 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c5c954-ab7b-4243-9f54-313bdecf7b29 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:36:26.675477 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5008e8e-9d52-4bfd-bcfd-b4fc78682827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:36:26.676098 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5008e8e-9d52-4bfd-bcfd-b4fc78682827 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:26.676384 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5008e8e-9d52-4bfd-bcfd-b4fc78682827 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:26.676660 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5008e8e-9d52-4bfd-bcfd-b4fc78682827 None admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:26.677356 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5008e8e-9d52-4bfd-bcfd-b4fc78682827 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:36:26.677693 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5008e8e-9d52-4bfd-bcfd-b4fc78682827 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:36:26.679063 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5008e8e-9d52-4bfd-bcfd-b4fc78682827 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:36:26.691969 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 606/1225] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Jul 26 13:36:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:36:26.697628 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fb8104-b6ff-47c9-bca6-4ef37a5dd200 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:36:26.800045 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:36:26.812236 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fb8104-b6ff-47c9-bca6-4ef37a5dd200 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:36:26.847886 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855c0a05-796c-481d-9000-86d40ed9caad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:36:26.848539 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855c0a05-796c-481d-9000-86d40ed9caad None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:26.848827 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855c0a05-796c-481d-9000-86d40ed9caad None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:26.849105 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855c0a05-796c-481d-9000-86d40ed9caad None admin] PATH_INFO: `/v3/users/placement` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:26.856919 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855c0a05-796c-481d-9000-86d40ed9caad None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:36:26.858463 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855c0a05-796c-481d-9000-86d40ed9caad None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:36:26.870557 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-855c0a05-796c-481d-9000-86d40ed9caad None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 26 13:36:26.870557 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:36:26.870557 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:36:26.870557 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:36:26.870557 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:36:26.870557 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:36:26.870557 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:36:26.870557 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:36:26.870557 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:36:26.870557 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:36:26.870557 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:36:26.870557 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:36:26.870557 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 13:36:26.870557 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 13:36:26.870557 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 13:36:26.870557 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 13:36:26.871736 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:36:26.871736 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:36:26.871736 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 13:36:26.871736 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 13:36:26.871736 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 13:36:26.871736 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 13:36:26.871736 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:36:26.871736 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:36:26.871736 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:36:26.871736 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:36:26.871736 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:36:26.871736 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:36:26.871736 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:36:26.871736 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:36:26.871736 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:36:26.871736 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:36:26.871736 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:36:26.871736 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:36:26.872721 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:36:26.872721 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:36:26.872721 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 13:36:26.872721 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 13:36:26.872721 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 13:36:26.872721 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 13:36:26.872721 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 13:36:26.872721 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 13:36:26.872721 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 26 13:36:26.872721 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:36:26.877379 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 620/1226] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:36:26 2022] GET /identity/v3/users/placement => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 13:36:26.878118 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5008e8e-9d52-4bfd-bcfd-b4fc78682827 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:36:26.931641 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:36:26.988704 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5008e8e-9d52-4bfd-bcfd-b4fc78682827 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:36:27.027505 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd6e2c8-8a53-4291-a3fd-cb15dd3618b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:36:27.028206 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd6e2c8-8a53-4291-a3fd-cb15dd3618b3 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:27.028518 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd6e2c8-8a53-4291-a3fd-cb15dd3618b3 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:27.028837 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd6e2c8-8a53-4291-a3fd-cb15dd3618b3 None admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:27.029752 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd6e2c8-8a53-4291-a3fd-cb15dd3618b3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:36:27.030158 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd6e2c8-8a53-4291-a3fd-cb15dd3618b3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:36:27.032343 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd6e2c8-8a53-4291-a3fd-cb15dd3618b3 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:36:27.052110 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 607/1227] 10.222.0.58 () {56 vars in 1143 bytes} [Tue Jul 26 13:36:26 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:36:27.057897 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855c0a05-796c-481d-9000-86d40ed9caad None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:36:27.168938 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855c0a05-796c-481d-9000-86d40ed9caad None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:36:27.208739 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d126530-9564-4dbd-b458-ec3d475e8716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:36:27.209387 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d126530-9564-4dbd-b458-ec3d475e8716 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:27.209772 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d126530-9564-4dbd-b458-ec3d475e8716 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:27.210049 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d126530-9564-4dbd-b458-ec3d475e8716 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:27.223886 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d126530-9564-4dbd-b458-ec3d475e8716 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:36:27.225947 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d126530-9564-4dbd-b458-ec3d475e8716 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:36:27.245331 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-3d126530-9564-4dbd-b458-ec3d475e8716 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:36:27.245331 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:36:27.245331 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:36:27.245331 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:36:27.245331 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:36:27.245331 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:36:27.245331 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:36:27.245331 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:36:27.245331 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:36:27.245331 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:36:27.245331 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:36:27.245331 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:36:27.245331 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 13:36:27.245331 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 13:36:27.245331 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 13:36:27.246307 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 13:36:27.246307 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:36:27.246307 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:36:27.246307 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:36:27.246307 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:36:27.246307 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:36:27.246307 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:36:27.246307 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:36:27.246307 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:36:27.246307 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:36:27.246307 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:36:27.246307 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:36:27.246307 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:36:27.246307 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:36:27.246307 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:36:27.246307 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:36:27.246307 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:36:27.247327 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 13:36:27.247327 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 13:36:27.247327 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:36:27.247327 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:36:27.252050 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 621/1228] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:36:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:36:27.253787 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd6e2c8-8a53-4291-a3fd-cb15dd3618b3 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:36:27.375133 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd6e2c8-8a53-4291-a3fd-cb15dd3618b3 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:36:27.378884 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:36:27.415877 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7cf133-77df-4306-b1e6-af73722163e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:36:27.416556 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7cf133-77df-4306-b1e6-af73722163e9 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:27.417141 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7cf133-77df-4306-b1e6-af73722163e9 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:27.417385 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7cf133-77df-4306-b1e6-af73722163e9 None admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:27.418169 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7cf133-77df-4306-b1e6-af73722163e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:36:27.418483 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7cf133-77df-4306-b1e6-af73722163e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:36:27.419966 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7cf133-77df-4306-b1e6-af73722163e9 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:36:27.435352 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 608/1229] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Jul 26 13:36:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:36:27.441917 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d126530-9564-4dbd-b458-ec3d475e8716 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:36:27.451952 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:36:27.556972 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d126530-9564-4dbd-b458-ec3d475e8716 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:36:27.594675 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caab5300-22bb-4a0f-85f8-1511bd14a9ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:36:27.595324 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caab5300-22bb-4a0f-85f8-1511bd14a9ef None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:27.595605 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caab5300-22bb-4a0f-85f8-1511bd14a9ef None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:27.595875 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caab5300-22bb-4a0f-85f8-1511bd14a9ef None admin] PATH_INFO: `/v3/projects/service` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:27.602565 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caab5300-22bb-4a0f-85f8-1511bd14a9ef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:36:27.603978 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caab5300-22bb-4a0f-85f8-1511bd14a9ef None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:36:27.615548 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-caab5300-22bb-4a0f-85f8-1511bd14a9ef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 13:36:27.615548 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:36:27.615548 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:36:27.615548 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:36:27.615548 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:36:27.615548 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:36:27.615548 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:36:27.615548 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:36:27.615548 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:36:27.615548 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:36:27.615548 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:36:27.615548 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:36:27.615548 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 13:36:27.615548 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 13:36:27.615548 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 13:36:27.616647 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 13:36:27.616647 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:36:27.616647 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:36:27.616647 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:36:27.616647 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:36:27.616647 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:36:27.616647 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:36:27.616647 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:36:27.616647 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:36:27.616647 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:36:27.616647 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:36:27.616647 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:36:27.616647 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:36:27.616647 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:36:27.616647 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:36:27.616647 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:36:27.616647 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:36:27.617805 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 13:36:27.617805 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 13:36:27.617805 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 13:36:27.617805 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 13:36:27.617805 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 13:36:27.617805 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 13:36:27.617805 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 13:36:27.617805 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:36:27.617805 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 622/1230] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Jul 26 13:36:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:36:27.622931 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7cf133-77df-4306-b1e6-af73722163e9 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:36:27.730896 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7cf133-77df-4306-b1e6-af73722163e9 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:36:27.767684 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6cfee4-a31f-461b-af51-85c900fe8e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:36:27.768308 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6cfee4-a31f-461b-af51-85c900fe8e3f None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:27.768591 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6cfee4-a31f-461b-af51-85c900fe8e3f None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:27.769031 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6cfee4-a31f-461b-af51-85c900fe8e3f None admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:27.769881 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6cfee4-a31f-461b-af51-85c900fe8e3f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:36:27.770191 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6cfee4-a31f-461b-af51-85c900fe8e3f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:36:27.771705 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6cfee4-a31f-461b-af51-85c900fe8e3f None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:36:27.785814 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 609/1231] 10.222.0.58 () {56 vars in 1148 bytes} [Tue Jul 26 13:36:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:36:27.792064 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caab5300-22bb-4a0f-85f8-1511bd14a9ef None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:36:27.868497 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:36:27.911848 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caab5300-22bb-4a0f-85f8-1511bd14a9ef None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:36:27.949827 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b179a791-39e3-4582-af72-2cb23f70c3a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:36:27.950510 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b179a791-39e3-4582-af72-2cb23f70c3a7 None admin] REQUEST_METHOD: `PUT` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:27.950789 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b179a791-39e3-4582-af72-2cb23f70c3a7 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:27.951064 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b179a791-39e3-4582-af72-2cb23f70c3a7 None admin] PATH_INFO: `/v3/projects/1e8bb4385c204210a1bc77b1f0d654f3/users/835cbc13756b41b598142df3e35f5492/roles/17e0f3c73a8a4a54b3e7b78878dfbf09` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:27.968689 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b179a791-39e3-4582-af72-2cb23f70c3a7 None admin] RBAC: Authorizing `identity:create_grant(project_id=1e8bb4385c204210a1bc77b1f0d654f3, user_id=835cbc13756b41b598142df3e35f5492, role_id=17e0f3c73a8a4a54b3e7b78878dfbf09)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:36:27.970184 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b179a791-39e3-4582-af72-2cb23f70c3a7 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:36:27.985723 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 623/1232] 10.222.0.58 () {58 vars in 1439 bytes} [Tue Jul 26 13:36:27 2022] PUT /identity/v3/projects/1e8bb4385c204210a1bc77b1f0d654f3/users/835cbc13756b41b598142df3e35f5492/roles/17e0f3c73a8a4a54b3e7b78878dfbf09 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 13:36:30.016846 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cfeea6-a003-4142-9101-379bef57ede8 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:30.017414 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cfeea6-a003-4142-9101-379bef57ede8 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:30.017589 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cfeea6-a003-4142-9101-379bef57ede8 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:30.018975 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 610/1233] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:36:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:36:30.024077 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06021eeb-784a-4c7a-9fd2-d812bb7e2a15 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:30.024604 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06021eeb-784a-4c7a-9fd2-d812bb7e2a15 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:30.024920 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06021eeb-784a-4c7a-9fd2-d812bb7e2a15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:30.049588 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-06021eeb-784a-4c7a-9fd2-d812bb7e2a15 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:36:30.050785 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-06021eeb-784a-4c7a-9fd2-d812bb7e2a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:36:30.121664 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 624/1234] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:36:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 13:36:30.147112 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cfeea6-a003-4142-9101-379bef57ede8 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:36:30.174590 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:36:30.270223 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cfeea6-a003-4142-9101-379bef57ede8 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:36:30.312124 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5641a878-b9b4-4444-83ec-dbca981a0cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:36:30.312790 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5641a878-b9b4-4444-83ec-dbca981a0cda None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:30.313077 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5641a878-b9b4-4444-83ec-dbca981a0cda None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:30.313377 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5641a878-b9b4-4444-83ec-dbca981a0cda None admin] PATH_INFO: `/v3/roles/service` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:30.326083 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5641a878-b9b4-4444-83ec-dbca981a0cda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:36:30.331701 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-5641a878-b9b4-4444-83ec-dbca981a0cda None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 13:36:30.331701 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:36:30.331701 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:36:30.331701 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:36:30.331701 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:36:30.331701 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:36:30.331701 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:36:30.331701 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:36:30.331701 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:36:30.331701 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:36:30.331701 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:36:30.331701 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:36:30.331701 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:36:30.331701 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:36:30.331701 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:36:30.331701 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:36:30.332696 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:36:30.332696 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:36:30.332696 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:36:30.332696 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:36:30.332696 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:36:30.332696 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:36:30.332696 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:36:30.332696 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:36:30.332696 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:36:30.332696 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:36:30.332696 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:36:30.332696 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:36:30.332696 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:36:30.332696 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:36:30.332696 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:36:30.332696 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:36:30.338851 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:36:30.338851 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:36:30.338851 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:36:30.338851 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:36:30.338851 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:36:30.338851 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:36:30.338851 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:36:30.338851 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:36:30.338851 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:36:30.338851 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:36:30.338851 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 13:36:30.338851 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:36:30.338851 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 611/1235] 10.222.0.58 () {56 vars in 1102 bytes} [Tue Jul 26 13:36:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:36:30.339958 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06021eeb-784a-4c7a-9fd2-d812bb7e2a15 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:36:30.394226 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:36:30.462155 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06021eeb-784a-4c7a-9fd2-d812bb7e2a15 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:36:30.511757 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc493992-0974-4095-97fa-6fff86cc1995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:36:30.512508 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc493992-0974-4095-97fa-6fff86cc1995 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:30.512787 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc493992-0974-4095-97fa-6fff86cc1995 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:30.513060 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc493992-0974-4095-97fa-6fff86cc1995 None admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:30.513819 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc493992-0974-4095-97fa-6fff86cc1995 None admin] RBAC: Adding query filter params (name=service) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:36:30.514124 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc493992-0974-4095-97fa-6fff86cc1995 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:36:30.515529 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc493992-0974-4095-97fa-6fff86cc1995 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:36:30.526539 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 625/1236] 10.222.0.58 () {56 vars in 1103 bytes} [Tue Jul 26 13:36:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:36:30.533712 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5641a878-b9b4-4444-83ec-dbca981a0cda None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:36:30.627838 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:36:30.645131 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5641a878-b9b4-4444-83ec-dbca981a0cda None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:36:30.680784 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5003123c-5674-40f7-bbf8-1d9870522d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:36:30.681437 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5003123c-5674-40f7-bbf8-1d9870522d58 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:30.681751 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5003123c-5674-40f7-bbf8-1d9870522d58 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:30.682026 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5003123c-5674-40f7-bbf8-1d9870522d58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:30.692447 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5003123c-5674-40f7-bbf8-1d9870522d58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:36:30.693917 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5003123c-5674-40f7-bbf8-1d9870522d58 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:36:30.709343 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-5003123c-5674-40f7-bbf8-1d9870522d58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:36:30.709343 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:36:30.709343 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:36:30.709343 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:36:30.709343 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:36:30.709343 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:36:30.709343 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:36:30.709343 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:36:30.709343 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:36:30.709343 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:36:30.709343 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:36:30.709343 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:36:30.709343 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 13:36:30.709343 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 13:36:30.709343 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 13:36:30.710439 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 13:36:30.710439 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:36:30.710439 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:36:30.710439 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:36:30.710439 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:36:30.710439 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:36:30.710439 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:36:30.710439 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:36:30.710439 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:36:30.710439 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:36:30.710439 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:36:30.710439 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:36:30.710439 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:36:30.710439 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:36:30.710439 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:36:30.710439 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:36:30.710439 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:36:30.711440 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 13:36:30.711440 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 13:36:30.711440 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:36:30.711440 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:36:30.711999 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 612/1237] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:36:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:36:30.716996 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc493992-0974-4095-97fa-6fff86cc1995 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:36:30.847505 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc493992-0974-4095-97fa-6fff86cc1995 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:36:30.857087 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:36:30.887308 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd125fc3-0c24-4db1-bf18-e457298347e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:36:30.888020 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd125fc3-0c24-4db1-bf18-e457298347e1 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:30.888399 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd125fc3-0c24-4db1-bf18-e457298347e1 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:30.888684 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd125fc3-0c24-4db1-bf18-e457298347e1 None admin] PATH_INFO: `/v3/domains` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:30.889467 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd125fc3-0c24-4db1-bf18-e457298347e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:36:30.889935 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd125fc3-0c24-4db1-bf18-e457298347e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:36:30.892109 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd125fc3-0c24-4db1-bf18-e457298347e1 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:36:30.906650 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 626/1238] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Jul 26 13:36:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:36:30.912516 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5003123c-5674-40f7-bbf8-1d9870522d58 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:36:31.026043 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5003123c-5674-40f7-bbf8-1d9870522d58 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:36:31.065288 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae30114-19e7-48c9-9b10-8b214ea46141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:36:31.066111 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae30114-19e7-48c9-9b10-8b214ea46141 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:31.066454 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae30114-19e7-48c9-9b10-8b214ea46141 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:31.066752 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae30114-19e7-48c9-9b10-8b214ea46141 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:31.069869 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:36:31.076292 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae30114-19e7-48c9-9b10-8b214ea46141 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:36:31.078212 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae30114-19e7-48c9-9b10-8b214ea46141 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:36:31.090716 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-4ae30114-19e7-48c9-9b10-8b214ea46141 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 26 13:36:31.090716 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:36:31.090716 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:36:31.090716 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:36:31.090716 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:36:31.090716 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:36:31.090716 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:36:31.090716 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:36:31.090716 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:36:31.090716 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:36:31.090716 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:36:31.090716 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:36:31.090716 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 13:36:31.090716 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 13:36:31.090716 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 13:36:31.090716 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 13:36:31.091899 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:36:31.091899 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:36:31.091899 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 13:36:31.091899 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 13:36:31.091899 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 13:36:31.091899 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 13:36:31.091899 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:36:31.091899 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:36:31.091899 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:36:31.091899 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:36:31.091899 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:36:31.091899 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:36:31.091899 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:36:31.091899 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:36:31.091899 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:36:31.091899 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:36:31.091899 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:36:31.091899 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:36:31.093010 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:36:31.093010 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:36:31.093010 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 13:36:31.093010 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 13:36:31.093010 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 13:36:31.093010 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 13:36:31.093010 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 13:36:31.093010 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 13:36:31.093010 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 26 13:36:31.093010 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:36:31.097709 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 613/1239] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:36:30 2022] GET /identity/v3/users/placement => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:36:31.099341 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd125fc3-0c24-4db1-bf18-e457298347e1 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:36:31.215944 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd125fc3-0c24-4db1-bf18-e457298347e1 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:36:31.256419 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696154c8-b664-458e-8bb3-42cce896b108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:36:31.257043 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696154c8-b664-458e-8bb3-42cce896b108 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:31.257319 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696154c8-b664-458e-8bb3-42cce896b108 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:31.257624 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696154c8-b664-458e-8bb3-42cce896b108 None admin] PATH_INFO: `/v3/users` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:31.259037 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696154c8-b664-458e-8bb3-42cce896b108 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:36:31.259348 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696154c8-b664-458e-8bb3-42cce896b108 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:36:31.260773 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696154c8-b664-458e-8bb3-42cce896b108 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:36:31.281427 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 627/1240] 10.222.0.58 () {56 vars in 1143 bytes} [Tue Jul 26 13:36:31 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:36:31.287660 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae30114-19e7-48c9-9b10-8b214ea46141 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:36:31.409849 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae30114-19e7-48c9-9b10-8b214ea46141 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:36:31.461029 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db859a07-aa7e-4a2d-915d-418682ed50b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:36:31.462251 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db859a07-aa7e-4a2d-915d-418682ed50b2 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:31.462411 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db859a07-aa7e-4a2d-915d-418682ed50b2 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:31.462706 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db859a07-aa7e-4a2d-915d-418682ed50b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:31.541205 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:36:31.550871 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db859a07-aa7e-4a2d-915d-418682ed50b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:36:31.552331 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db859a07-aa7e-4a2d-915d-418682ed50b2 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:36:31.665198 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 614/1241] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Jul 26 13:36:31 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 26 13:36:31.672339 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696154c8-b664-458e-8bb3-42cce896b108 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:36:31.702616 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:36:31.798398 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696154c8-b664-458e-8bb3-42cce896b108 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:36:31.839294 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb91a9e-1b48-4a1a-9df8-69382e99db73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:36:31.839951 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb91a9e-1b48-4a1a-9df8-69382e99db73 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:31.841824 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb91a9e-1b48-4a1a-9df8-69382e99db73 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:31.842121 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb91a9e-1b48-4a1a-9df8-69382e99db73 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:31.853753 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb91a9e-1b48-4a1a-9df8-69382e99db73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:36:31.855443 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb91a9e-1b48-4a1a-9df8-69382e99db73 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:36:31.872787 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-3cb91a9e-1b48-4a1a-9df8-69382e99db73 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:36:31.872787 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:36:31.872787 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:36:31.872787 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:36:31.872787 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:36:31.872787 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:36:31.872787 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:36:31.872787 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:36:31.872787 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:36:31.872787 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:36:31.872787 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:36:31.872787 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:36:31.872787 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 13:36:31.872787 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 13:36:31.872787 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 13:36:31.873930 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 13:36:31.873930 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:36:31.873930 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:36:31.873930 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:36:31.873930 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:36:31.873930 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:36:31.873930 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:36:31.873930 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:36:31.873930 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:36:31.873930 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:36:31.873930 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:36:31.873930 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:36:31.873930 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:36:31.873930 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:36:31.873930 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:36:31.873930 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:36:31.873930 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:36:31.881541 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 13:36:31.881541 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 13:36:31.881541 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:36:31.881541 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:36:31.881541 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 628/1242] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:36:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:36:31.882716 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db859a07-aa7e-4a2d-915d-418682ed50b2 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:36:31.988962 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:36:31.995321 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db859a07-aa7e-4a2d-915d-418682ed50b2 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:36:32.034181 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef63b2e6-c22e-4e85-8665-f18e70f11f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:36:32.035587 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef63b2e6-c22e-4e85-8665-f18e70f11f0b None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:32.036678 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef63b2e6-c22e-4e85-8665-f18e70f11f0b None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:32.036960 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef63b2e6-c22e-4e85-8665-f18e70f11f0b None admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:32.037729 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef63b2e6-c22e-4e85-8665-f18e70f11f0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:36:32.038040 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef63b2e6-c22e-4e85-8665-f18e70f11f0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:36:32.039466 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef63b2e6-c22e-4e85-8665-f18e70f11f0b None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:36:32.052518 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 615/1243] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Jul 26 13:36:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:36:32.058377 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb91a9e-1b48-4a1a-9df8-69382e99db73 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:36:32.144879 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:36:32.167252 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb91a9e-1b48-4a1a-9df8-69382e99db73 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:36:32.204639 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42b914f-41fe-4f51-b346-23f5093af7fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:36:32.205225 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42b914f-41fe-4f51-b346-23f5093af7fc None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:32.205560 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42b914f-41fe-4f51-b346-23f5093af7fc None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:32.205818 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42b914f-41fe-4f51-b346-23f5093af7fc None admin] PATH_INFO: `/v3/projects/service` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:32.212668 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42b914f-41fe-4f51-b346-23f5093af7fc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:36:32.214137 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42b914f-41fe-4f51-b346-23f5093af7fc None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:36:32.226170 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-d42b914f-41fe-4f51-b346-23f5093af7fc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 13:36:32.226170 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:36:32.226170 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:36:32.226170 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:36:32.226170 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:36:32.226170 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:36:32.226170 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:36:32.226170 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:36:32.226170 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:36:32.226170 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:36:32.226170 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:36:32.226170 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:36:32.226170 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 13:36:32.226170 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 13:36:32.226170 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 13:36:32.227131 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 13:36:32.227131 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:36:32.227131 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:36:32.227131 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:36:32.227131 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:36:32.227131 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:36:32.227131 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:36:32.227131 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:36:32.227131 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:36:32.227131 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:36:32.227131 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:36:32.227131 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:36:32.227131 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:36:32.227131 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:36:32.227131 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:36:32.227131 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:36:32.227131 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:36:32.228034 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 13:36:32.228034 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 13:36:32.228034 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 13:36:32.228034 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 13:36:32.228034 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 13:36:32.228034 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 13:36:32.228034 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 13:36:32.228034 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:36:32.228034 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 629/1244] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Jul 26 13:36:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:36:32.233537 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef63b2e6-c22e-4e85-8665-f18e70f11f0b None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:36:32.345114 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef63b2e6-c22e-4e85-8665-f18e70f11f0b None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:36:32.383552 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d247716e-ba22-4937-a68f-95f493734cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:36:32.384189 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d247716e-ba22-4937-a68f-95f493734cb4 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:32.384469 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d247716e-ba22-4937-a68f-95f493734cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:32.384744 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d247716e-ba22-4937-a68f-95f493734cb4 None admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:32.385518 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d247716e-ba22-4937-a68f-95f493734cb4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:36:32.385834 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d247716e-ba22-4937-a68f-95f493734cb4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:36:32.387270 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d247716e-ba22-4937-a68f-95f493734cb4 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:36:32.400749 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 616/1245] 10.222.0.58 () {56 vars in 1148 bytes} [Tue Jul 26 13:36:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:36:32.407283 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42b914f-41fe-4f51-b346-23f5093af7fc None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:36:32.517451 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42b914f-41fe-4f51-b346-23f5093af7fc None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:36:32.558357 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85a2e00-e8a8-4571-876d-fa9a2cce493b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:36:32.559057 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85a2e00-e8a8-4571-876d-fa9a2cce493b None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:32.559339 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85a2e00-e8a8-4571-876d-fa9a2cce493b None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:32.559614 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85a2e00-e8a8-4571-876d-fa9a2cce493b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:32.560412 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85a2e00-e8a8-4571-876d-fa9a2cce493b None admin] RBAC: Adding query filter params (role.id=17e0f3c73a8a4a54b3e7b78878dfbf09, scope.project.id=1e8bb4385c204210a1bc77b1f0d654f3, user.id=835cbc13756b41b598142df3e35f5492) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:36:32.560721 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85a2e00-e8a8-4571-876d-fa9a2cce493b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:36:32.562913 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85a2e00-e8a8-4571-876d-fa9a2cce493b None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:36:32.569601 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 630/1246] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Jul 26 13:36:32 2022] GET /identity/v3/role_assignments?user.id=835cbc13756b41b598142df3e35f5492&scope.project.id=1e8bb4385c204210a1bc77b1f0d654f3&role.id=17e0f3c73a8a4a54b3e7b78878dfbf09 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:36:34.801965 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da4ec0c-1836-4bc9-b6d2-bf19a6798ae8 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:34.802590 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da4ec0c-1836-4bc9-b6d2-bf19a6798ae8 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:34.802710 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da4ec0c-1836-4bc9-b6d2-bf19a6798ae8 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:34.804174 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 617/1247] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:36:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:36:34.810405 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa2f7c1-d1cd-4788-9efa-d302955e20f1 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:34.810954 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa2f7c1-d1cd-4788-9efa-d302955e20f1 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:34.811074 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa2f7c1-d1cd-4788-9efa-d302955e20f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:34.818544 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:36:34.834219 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-1fa2f7c1-d1cd-4788-9efa-d302955e20f1 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:36:34.835678 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-1fa2f7c1-d1cd-4788-9efa-d302955e20f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:36:34.915250 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 631/1248] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:36:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 13:36:34.939499 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da4ec0c-1836-4bc9-b6d2-bf19a6798ae8 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:36:34.959304 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:36:35.059722 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da4ec0c-1836-4bc9-b6d2-bf19a6798ae8 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:36:35.097372 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f6d075-2d6c-49ae-8033-0c5d9be6a106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:36:35.098050 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f6d075-2d6c-49ae-8033-0c5d9be6a106 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:35.098335 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f6d075-2d6c-49ae-8033-0c5d9be6a106 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:35.098613 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f6d075-2d6c-49ae-8033-0c5d9be6a106 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:35.112106 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f6d075-2d6c-49ae-8033-0c5d9be6a106 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:36:35.117721 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-a8f6d075-2d6c-49ae-8033-0c5d9be6a106 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 13:36:35.117721 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:36:35.117721 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:36:35.117721 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:36:35.117721 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:36:35.117721 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:36:35.117721 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:36:35.117721 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:36:35.117721 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:36:35.117721 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:36:35.117721 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:36:35.117721 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:36:35.117721 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:36:35.117721 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:36:35.117721 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:36:35.117721 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:36:35.118920 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:36:35.118920 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:36:35.118920 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:36:35.118920 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:36:35.118920 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:36:35.118920 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:36:35.118920 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:36:35.118920 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:36:35.118920 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:36:35.118920 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:36:35.118920 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:36:35.118920 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:36:35.118920 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:36:35.118920 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:36:35.118920 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:36:35.118920 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:36:35.119792 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:36:35.119792 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:36:35.119792 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:36:35.119792 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:36:35.119792 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:36:35.119792 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:36:35.119792 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:36:35.119792 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:36:35.119792 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:36:35.119792 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:36:35.119792 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 13:36:35.119792 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:36:35.119792 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 618/1249] 10.222.0.58 () {56 vars in 1096 bytes} [Tue Jul 26 13:36:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:36:35.125880 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa2f7c1-d1cd-4788-9efa-d302955e20f1 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:36:35.241086 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa2f7c1-d1cd-4788-9efa-d302955e20f1 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:36:35.275439 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:36:35.282301 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84214fff-9c59-4c2d-bed8-74d16a079a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:36:35.282926 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84214fff-9c59-4c2d-bed8-74d16a079a2e None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:35.283279 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84214fff-9c59-4c2d-bed8-74d16a079a2e None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:35.283618 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84214fff-9c59-4c2d-bed8-74d16a079a2e None admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:35.284420 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84214fff-9c59-4c2d-bed8-74d16a079a2e None admin] RBAC: Adding query filter params (name=admin) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:36:35.284756 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84214fff-9c59-4c2d-bed8-74d16a079a2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:36:35.286340 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84214fff-9c59-4c2d-bed8-74d16a079a2e None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:36:35.296455 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 632/1250] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Jul 26 13:36:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:36:35.302384 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f6d075-2d6c-49ae-8033-0c5d9be6a106 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:36:35.394491 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:36:35.421719 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f6d075-2d6c-49ae-8033-0c5d9be6a106 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:36:35.461911 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9886b85c-9ecc-46ee-97c1-1270c2eaf0b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:36:35.462689 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9886b85c-9ecc-46ee-97c1-1270c2eaf0b7 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:35.462996 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9886b85c-9ecc-46ee-97c1-1270c2eaf0b7 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:35.463275 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9886b85c-9ecc-46ee-97c1-1270c2eaf0b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:35.476509 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9886b85c-9ecc-46ee-97c1-1270c2eaf0b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:36:35.478048 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9886b85c-9ecc-46ee-97c1-1270c2eaf0b7 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:36:35.494832 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-9886b85c-9ecc-46ee-97c1-1270c2eaf0b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:36:35.494832 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:36:35.494832 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:36:35.494832 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:36:35.494832 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:36:35.494832 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:36:35.494832 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:36:35.494832 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:36:35.494832 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:36:35.494832 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:36:35.494832 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:36:35.494832 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:36:35.494832 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 13:36:35.494832 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 13:36:35.494832 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 13:36:35.496073 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 13:36:35.496073 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:36:35.496073 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:36:35.496073 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:36:35.496073 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:36:35.496073 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:36:35.496073 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:36:35.496073 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:36:35.496073 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:36:35.496073 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:36:35.496073 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:36:35.496073 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:36:35.496073 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:36:35.496073 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:36:35.496073 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:36:35.496073 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:36:35.496073 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:36:35.497188 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 13:36:35.497188 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 13:36:35.497188 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:36:35.497188 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:36:35.501677 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 619/1251] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:36:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 13:36:35.502727 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84214fff-9c59-4c2d-bed8-74d16a079a2e None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:36:35.617467 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84214fff-9c59-4c2d-bed8-74d16a079a2e None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:36:35.656546 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d714fbe-978d-4dea-b5c3-9b1c8a4e0ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:36:35.657177 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d714fbe-978d-4dea-b5c3-9b1c8a4e0ce9 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:35.657557 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d714fbe-978d-4dea-b5c3-9b1c8a4e0ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:35.657842 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d714fbe-978d-4dea-b5c3-9b1c8a4e0ce9 None admin] PATH_INFO: `/v3/domains` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:35.658665 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d714fbe-978d-4dea-b5c3-9b1c8a4e0ce9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:36:35.659012 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d714fbe-978d-4dea-b5c3-9b1c8a4e0ce9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:36:35.660375 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d714fbe-978d-4dea-b5c3-9b1c8a4e0ce9 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:36:35.679021 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 633/1252] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Jul 26 13:36:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:36:35.680743 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9886b85c-9ecc-46ee-97c1-1270c2eaf0b7 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:36:35.801007 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9886b85c-9ecc-46ee-97c1-1270c2eaf0b7 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:36:35.835787 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:36:35.844199 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8cb897-ca58-4e7b-9a4b-48f7a0c1e1e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:36:35.844837 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8cb897-ca58-4e7b-9a4b-48f7a0c1e1e3 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:35.845117 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8cb897-ca58-4e7b-9a4b-48f7a0c1e1e3 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:35.845387 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8cb897-ca58-4e7b-9a4b-48f7a0c1e1e3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:35.853923 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8cb897-ca58-4e7b-9a4b-48f7a0c1e1e3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:36:35.855501 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8cb897-ca58-4e7b-9a4b-48f7a0c1e1e3 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:36:35.867521 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-8c8cb897-ca58-4e7b-9a4b-48f7a0c1e1e3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 26 13:36:35.867521 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:36:35.867521 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:36:35.867521 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:36:35.867521 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:36:35.867521 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:36:35.867521 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:36:35.867521 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:36:35.867521 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:36:35.867521 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:36:35.867521 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:36:35.867521 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:36:35.867521 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 13:36:35.867521 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 13:36:35.867521 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 13:36:35.867521 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 13:36:35.868494 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:36:35.868494 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:36:35.868494 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 13:36:35.868494 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 13:36:35.868494 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 13:36:35.868494 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 13:36:35.868494 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:36:35.868494 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:36:35.868494 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:36:35.868494 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:36:35.868494 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:36:35.868494 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:36:35.868494 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:36:35.868494 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:36:35.868494 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:36:35.868494 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:36:35.868494 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:36:35.868494 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:36:35.869429 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:36:35.869429 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:36:35.869429 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 13:36:35.869429 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 13:36:35.869429 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 13:36:35.869429 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 13:36:35.869429 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 13:36:35.869429 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 13:36:35.869429 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 26 13:36:35.869429 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:36:35.869429 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 620/1253] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:36:35 2022] GET /identity/v3/users/placement => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:36:35.875090 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d714fbe-978d-4dea-b5c3-9b1c8a4e0ce9 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:36:35.941751 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:36:35.995340 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d714fbe-978d-4dea-b5c3-9b1c8a4e0ce9 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:36:36.038849 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e30daf4-5a1a-49ae-810d-e232cac272b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:36:36.039503 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e30daf4-5a1a-49ae-810d-e232cac272b4 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:36.039786 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e30daf4-5a1a-49ae-810d-e232cac272b4 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:36.040059 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e30daf4-5a1a-49ae-810d-e232cac272b4 None admin] PATH_INFO: `/v3/users` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:36.040850 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e30daf4-5a1a-49ae-810d-e232cac272b4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:36:36.041161 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e30daf4-5a1a-49ae-810d-e232cac272b4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:36:36.042859 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e30daf4-5a1a-49ae-810d-e232cac272b4 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:36:36.061044 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 634/1254] 10.222.0.58 () {56 vars in 1143 bytes} [Tue Jul 26 13:36:35 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:36:36.067393 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8cb897-ca58-4e7b-9a4b-48f7a0c1e1e3 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:36:36.188270 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8cb897-ca58-4e7b-9a4b-48f7a0c1e1e3 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:36:36.227843 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fda396-bb3c-4d3f-9a66-f74f7453c15c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:36:36.228528 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fda396-bb3c-4d3f-9a66-f74f7453c15c None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:36.228808 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fda396-bb3c-4d3f-9a66-f74f7453c15c None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:36.229079 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fda396-bb3c-4d3f-9a66-f74f7453c15c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:36.294420 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:36:36.309594 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fda396-bb3c-4d3f-9a66-f74f7453c15c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:36:36.311150 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fda396-bb3c-4d3f-9a66-f74f7453c15c None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:36:36.423574 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 621/1255] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Jul 26 13:36:36 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 13:36:36.429578 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e30daf4-5a1a-49ae-810d-e232cac272b4 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:36:36.535388 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e30daf4-5a1a-49ae-810d-e232cac272b4 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:36:36.549171 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:36:36.571524 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a38b0e7-4fa8-4c8b-bf67-943869157b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:36:36.572210 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a38b0e7-4fa8-4c8b-bf67-943869157b84 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:36.572495 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a38b0e7-4fa8-4c8b-bf67-943869157b84 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:36.572772 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a38b0e7-4fa8-4c8b-bf67-943869157b84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:36.583833 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a38b0e7-4fa8-4c8b-bf67-943869157b84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:36:36.585380 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a38b0e7-4fa8-4c8b-bf67-943869157b84 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:36:36.602319 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-5a38b0e7-4fa8-4c8b-bf67-943869157b84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:36:36.602319 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:36:36.602319 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:36:36.602319 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:36:36.602319 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:36:36.602319 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:36:36.602319 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:36:36.602319 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:36:36.602319 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:36:36.602319 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:36:36.602319 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:36:36.602319 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:36:36.602319 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 13:36:36.602319 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 13:36:36.602319 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 13:36:36.603379 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 13:36:36.603379 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:36:36.603379 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:36:36.603379 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:36:36.603379 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:36:36.603379 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:36:36.603379 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:36:36.603379 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:36:36.603379 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:36:36.603379 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:36:36.603379 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:36:36.603379 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:36:36.603379 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:36:36.603379 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:36:36.603379 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:36:36.603379 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:36:36.603379 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:36:36.604336 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 13:36:36.604336 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 13:36:36.604336 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:36:36.604336 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:36:36.604336 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 635/1256] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:36:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:36:36.610101 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fda396-bb3c-4d3f-9a66-f74f7453c15c None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:36:36.704833 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:36:36.718985 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fda396-bb3c-4d3f-9a66-f74f7453c15c None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:36:36.756692 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d6c126-f63b-4e09-a32e-be794e0bd263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:36:36.757459 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d6c126-f63b-4e09-a32e-be794e0bd263 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:36.757806 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d6c126-f63b-4e09-a32e-be794e0bd263 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:36.758118 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d6c126-f63b-4e09-a32e-be794e0bd263 None admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:36.758934 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d6c126-f63b-4e09-a32e-be794e0bd263 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:36:36.759288 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d6c126-f63b-4e09-a32e-be794e0bd263 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:36:36.760880 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d6c126-f63b-4e09-a32e-be794e0bd263 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:36:36.775664 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 622/1257] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Jul 26 13:36:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:36:36.782977 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a38b0e7-4fa8-4c8b-bf67-943869157b84 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:36:36.895812 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a38b0e7-4fa8-4c8b-bf67-943869157b84 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:36:36.932669 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38af686-d36e-4f11-82e7-249f2aa91410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:36:36.933318 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38af686-d36e-4f11-82e7-249f2aa91410 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:36.933633 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38af686-d36e-4f11-82e7-249f2aa91410 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:36.933912 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38af686-d36e-4f11-82e7-249f2aa91410 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:36.940955 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38af686-d36e-4f11-82e7-249f2aa91410 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:36:36.942595 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38af686-d36e-4f11-82e7-249f2aa91410 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:36:36.954835 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-e38af686-d36e-4f11-82e7-249f2aa91410 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 13:36:36.954835 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:36:36.954835 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:36:36.954835 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:36:36.954835 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:36:36.954835 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:36:36.954835 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:36:36.954835 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:36:36.954835 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:36:36.954835 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:36:36.954835 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:36:36.954835 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:36:36.954835 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 13:36:36.954835 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 13:36:36.954835 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 13:36:36.955977 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 13:36:36.955977 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:36:36.955977 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:36:36.955977 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:36:36.955977 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:36:36.955977 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:36:36.955977 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:36:36.955977 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:36:36.955977 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:36:36.955977 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:36:36.955977 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:36:36.955977 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:36:36.955977 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:36:36.955977 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:36:36.955977 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:36:36.955977 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:36:36.955977 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:36:36.956916 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 13:36:36.956916 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 13:36:36.956916 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 13:36:36.956916 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 13:36:36.956916 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 13:36:36.956916 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 13:36:36.956916 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 13:36:36.956916 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:36:36.956916 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 636/1258] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Jul 26 13:36:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:36:36.962056 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d6c126-f63b-4e09-a32e-be794e0bd263 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:36:37.069038 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d6c126-f63b-4e09-a32e-be794e0bd263 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:36:37.107186 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0add4d-fbce-4ae0-b664-20b41e4dc647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:36:37.107819 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0add4d-fbce-4ae0-b664-20b41e4dc647 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:37.108100 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0add4d-fbce-4ae0-b664-20b41e4dc647 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:37.108373 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0add4d-fbce-4ae0-b664-20b41e4dc647 None admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:37.109281 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0add4d-fbce-4ae0-b664-20b41e4dc647 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:36:37.109622 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0add4d-fbce-4ae0-b664-20b41e4dc647 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:36:37.111061 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0add4d-fbce-4ae0-b664-20b41e4dc647 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:36:37.124946 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 623/1259] 10.222.0.58 () {56 vars in 1148 bytes} [Tue Jul 26 13:36:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:36:37.131366 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38af686-d36e-4f11-82e7-249f2aa91410 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:36:37.160961 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:36:37.250366 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38af686-d36e-4f11-82e7-249f2aa91410 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:36:37.287938 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb342906-1197-4ede-839f-6352913d2a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:36:37.288560 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb342906-1197-4ede-839f-6352913d2a80 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:37.288837 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb342906-1197-4ede-839f-6352913d2a80 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:37.289127 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb342906-1197-4ede-839f-6352913d2a80 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:37.290250 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb342906-1197-4ede-839f-6352913d2a80 None admin] RBAC: Adding query filter params (role.id=6e0201a9a6484f7d8b112b1eafff70bd, scope.project.id=1e8bb4385c204210a1bc77b1f0d654f3, user.id=835cbc13756b41b598142df3e35f5492) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:36:37.290569 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb342906-1197-4ede-839f-6352913d2a80 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:36:37.292705 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb342906-1197-4ede-839f-6352913d2a80 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:36:37.298434 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 637/1260] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Jul 26 13:36:37 2022] GET /identity/v3/role_assignments?user.id=835cbc13756b41b598142df3e35f5492&scope.project.id=1e8bb4385c204210a1bc77b1f0d654f3&role.id=6e0201a9a6484f7d8b112b1eafff70bd => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:36:39.288374 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5e87ac-c2ac-4b90-a88b-e22cde3256a7 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:39.288874 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5e87ac-c2ac-4b90-a88b-e22cde3256a7 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:39.288874 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5e87ac-c2ac-4b90-a88b-e22cde3256a7 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:39.290243 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 624/1261] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:36:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:36:39.295533 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9583c69-48e6-453c-8155-f881ec97086c None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:39.295916 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9583c69-48e6-453c-8155-f881ec97086c None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:39.296643 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9583c69-48e6-453c-8155-f881ec97086c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:39.314188 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-d9583c69-48e6-453c-8155-f881ec97086c None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:36:39.315355 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-d9583c69-48e6-453c-8155-f881ec97086c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:36:39.380502 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 638/1262] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:36:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 13:36:39.402185 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5e87ac-c2ac-4b90-a88b-e22cde3256a7 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:36:39.413344 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:36:39.502783 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5e87ac-c2ac-4b90-a88b-e22cde3256a7 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:36:39.535798 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e55da03-73e0-4a48-aa3a-bab589a40eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:36:39.536426 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e55da03-73e0-4a48-aa3a-bab589a40eb1 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:39.536712 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e55da03-73e0-4a48-aa3a-bab589a40eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:39.536981 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e55da03-73e0-4a48-aa3a-bab589a40eb1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:39.547917 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e55da03-73e0-4a48-aa3a-bab589a40eb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:36:39.552811 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-9e55da03-73e0-4a48-aa3a-bab589a40eb1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 13:36:39.552811 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:36:39.552811 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:36:39.552811 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:36:39.552811 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:36:39.552811 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:36:39.552811 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:36:39.552811 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:36:39.552811 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:36:39.552811 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:36:39.552811 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:36:39.552811 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:36:39.552811 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:36:39.552811 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:36:39.552811 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:36:39.552811 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:36:39.554024 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:36:39.554024 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:36:39.554024 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:36:39.554024 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:36:39.554024 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:36:39.554024 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:36:39.554024 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:36:39.554024 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:36:39.554024 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:36:39.554024 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:36:39.554024 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:36:39.554024 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:36:39.554024 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:36:39.554024 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:36:39.554024 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:36:39.554024 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:36:39.555261 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:36:39.555261 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:36:39.555261 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:36:39.555261 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:36:39.555261 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:36:39.555261 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:36:39.555261 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:36:39.555261 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:36:39.555261 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:36:39.555261 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:36:39.555261 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 13:36:39.555261 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:36:39.555261 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 625/1263] 10.222.0.58 () {56 vars in 1096 bytes} [Tue Jul 26 13:36:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:36:39.559935 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9583c69-48e6-453c-8155-f881ec97086c None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:36:39.577556 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:36:39.656091 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9583c69-48e6-453c-8155-f881ec97086c None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:36:39.689128 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55fafad-bc49-4ccc-b72d-a1349b64d00c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:36:39.689764 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55fafad-bc49-4ccc-b72d-a1349b64d00c None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:39.690045 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55fafad-bc49-4ccc-b72d-a1349b64d00c None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:39.690317 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55fafad-bc49-4ccc-b72d-a1349b64d00c None admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:39.691044 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55fafad-bc49-4ccc-b72d-a1349b64d00c None admin] RBAC: Adding query filter params (name=admin) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:36:39.691324 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55fafad-bc49-4ccc-b72d-a1349b64d00c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:36:39.692669 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55fafad-bc49-4ccc-b72d-a1349b64d00c None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:36:39.700368 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 639/1264] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Jul 26 13:36:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:36:39.706724 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e55da03-73e0-4a48-aa3a-bab589a40eb1 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:36:39.776988 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:36:39.813969 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e55da03-73e0-4a48-aa3a-bab589a40eb1 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:36:39.848075 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9118f1-52a1-4133-a464-67d4679dc074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:36:39.848725 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9118f1-52a1-4133-a464-67d4679dc074 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:39.849044 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9118f1-52a1-4133-a464-67d4679dc074 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:39.849433 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9118f1-52a1-4133-a464-67d4679dc074 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:39.859437 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9118f1-52a1-4133-a464-67d4679dc074 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:36:39.860897 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9118f1-52a1-4133-a464-67d4679dc074 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:36:39.875307 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-da9118f1-52a1-4133-a464-67d4679dc074 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:36:39.875307 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:36:39.875307 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:36:39.875307 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:36:39.875307 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:36:39.875307 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:36:39.875307 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:36:39.875307 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:36:39.875307 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:36:39.875307 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:36:39.875307 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:36:39.875307 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:36:39.875307 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 13:36:39.875307 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 13:36:39.875307 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 13:36:39.876347 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 13:36:39.876347 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:36:39.876347 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:36:39.876347 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:36:39.876347 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:36:39.876347 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:36:39.876347 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:36:39.876347 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:36:39.876347 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:36:39.876347 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:36:39.876347 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:36:39.876347 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:36:39.876347 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:36:39.876347 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:36:39.876347 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:36:39.876347 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:36:39.876347 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:36:39.877439 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 13:36:39.877439 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 13:36:39.877439 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:36:39.877439 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:36:39.877439 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 626/1265] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:36:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:36:39.882209 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55fafad-bc49-4ccc-b72d-a1349b64d00c None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:36:39.962636 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:36:39.982107 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55fafad-bc49-4ccc-b72d-a1349b64d00c None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:36:40.014262 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deccbcd8-a54d-43df-893d-1e20154f11ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:36:40.015390 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deccbcd8-a54d-43df-893d-1e20154f11ab None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:40.015672 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deccbcd8-a54d-43df-893d-1e20154f11ab None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:40.015945 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deccbcd8-a54d-43df-893d-1e20154f11ab None admin] PATH_INFO: `/v3/domains` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:40.016627 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deccbcd8-a54d-43df-893d-1e20154f11ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:36:40.016933 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deccbcd8-a54d-43df-893d-1e20154f11ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:36:40.018307 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deccbcd8-a54d-43df-893d-1e20154f11ab None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:36:40.029739 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 640/1266] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Jul 26 13:36:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:36:40.035941 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9118f1-52a1-4133-a464-67d4679dc074 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:36:40.132692 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9118f1-52a1-4133-a464-67d4679dc074 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:36:40.151932 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:36:40.165544 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185570b1-2384-46ba-9788-1227887e14a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:36:40.166171 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185570b1-2384-46ba-9788-1227887e14a5 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:40.166705 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185570b1-2384-46ba-9788-1227887e14a5 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:40.166979 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185570b1-2384-46ba-9788-1227887e14a5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:40.174189 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185570b1-2384-46ba-9788-1227887e14a5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:36:40.175624 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185570b1-2384-46ba-9788-1227887e14a5 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:36:40.187062 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-185570b1-2384-46ba-9788-1227887e14a5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 26 13:36:40.187062 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:36:40.187062 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:36:40.187062 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:36:40.187062 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:36:40.187062 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:36:40.187062 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:36:40.187062 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:36:40.187062 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:36:40.187062 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:36:40.187062 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:36:40.187062 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:36:40.187062 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 13:36:40.187062 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 13:36:40.187062 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 13:36:40.187062 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 13:36:40.188173 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:36:40.188173 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:36:40.188173 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 13:36:40.188173 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 13:36:40.188173 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 13:36:40.188173 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 13:36:40.188173 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:36:40.188173 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:36:40.188173 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:36:40.188173 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:36:40.188173 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:36:40.188173 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:36:40.188173 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:36:40.188173 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:36:40.188173 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:36:40.188173 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:36:40.188173 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:36:40.188173 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:36:40.189520 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:36:40.189520 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:36:40.189520 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 13:36:40.189520 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 13:36:40.189520 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 13:36:40.189520 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 13:36:40.189520 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 13:36:40.189520 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 13:36:40.189520 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 26 13:36:40.189520 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:36:40.189520 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 627/1267] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:36:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:36:40.193980 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deccbcd8-a54d-43df-893d-1e20154f11ab None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:36:40.288130 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deccbcd8-a54d-43df-893d-1e20154f11ab None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:36:40.320598 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60b4cf7-5cf5-4490-9379-4442507420bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:36:40.321209 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60b4cf7-5cf5-4490-9379-4442507420bd None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:40.321601 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60b4cf7-5cf5-4490-9379-4442507420bd None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:40.321792 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60b4cf7-5cf5-4490-9379-4442507420bd None admin] PATH_INFO: `/v3/users` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:40.322570 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60b4cf7-5cf5-4490-9379-4442507420bd None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:36:40.322876 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60b4cf7-5cf5-4490-9379-4442507420bd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:36:40.324284 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60b4cf7-5cf5-4490-9379-4442507420bd None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:36:40.333249 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:36:40.341741 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 641/1268] 10.222.0.58 () {56 vars in 1143 bytes} [Tue Jul 26 13:36:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:36:40.347382 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185570b1-2384-46ba-9788-1227887e14a5 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:36:40.446016 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185570b1-2384-46ba-9788-1227887e14a5 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:36:40.478021 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873c38ca-cce4-44d0-8e66-c135fd64e477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:36:40.478647 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873c38ca-cce4-44d0-8e66-c135fd64e477 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:40.478928 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873c38ca-cce4-44d0-8e66-c135fd64e477 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:40.479202 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873c38ca-cce4-44d0-8e66-c135fd64e477 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:40.489951 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873c38ca-cce4-44d0-8e66-c135fd64e477 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:36:40.491370 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873c38ca-cce4-44d0-8e66-c135fd64e477 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:36:40.505680 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-873c38ca-cce4-44d0-8e66-c135fd64e477 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:36:40.505680 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:36:40.505680 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:36:40.505680 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:36:40.505680 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:36:40.505680 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:36:40.505680 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:36:40.505680 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:36:40.505680 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:36:40.505680 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:36:40.505680 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:36:40.505680 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:36:40.505680 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 13:36:40.505680 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 13:36:40.505680 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 13:36:40.506711 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 13:36:40.506711 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:36:40.506711 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:36:40.506711 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:36:40.506711 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:36:40.506711 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:36:40.506711 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:36:40.506711 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:36:40.506711 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:36:40.506711 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:36:40.506711 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:36:40.506711 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:36:40.506711 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:36:40.506711 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:36:40.506711 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:36:40.506711 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:36:40.506711 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:36:40.507841 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 13:36:40.507841 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 13:36:40.507841 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:36:40.507841 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:36:40.509623 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 628/1269] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:36:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:36:40.512301 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60b4cf7-5cf5-4490-9379-4442507420bd None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:36:40.608318 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60b4cf7-5cf5-4490-9379-4442507420bd None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:36:40.642575 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c60ec0-a730-4bdd-84f2-6b27ef513171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:36:40.643182 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c60ec0-a730-4bdd-84f2-6b27ef513171 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:40.643466 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c60ec0-a730-4bdd-84f2-6b27ef513171 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:40.643741 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c60ec0-a730-4bdd-84f2-6b27ef513171 None admin] PATH_INFO: `/v3/domains` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:40.644425 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c60ec0-a730-4bdd-84f2-6b27ef513171 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:36:40.644732 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c60ec0-a730-4bdd-84f2-6b27ef513171 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:36:40.646196 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c60ec0-a730-4bdd-84f2-6b27ef513171 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:36:40.657056 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 642/1270] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Jul 26 13:36:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:36:40.662444 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873c38ca-cce4-44d0-8e66-c135fd64e477 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:36:40.694584 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:36:40.770031 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873c38ca-cce4-44d0-8e66-c135fd64e477 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:36:40.811990 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13222cd-93f8-40ee-9f95-d66cb2ed019e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:36:40.812623 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13222cd-93f8-40ee-9f95-d66cb2ed019e None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:40.812900 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13222cd-93f8-40ee-9f95-d66cb2ed019e None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:40.813170 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13222cd-93f8-40ee-9f95-d66cb2ed019e None admin] PATH_INFO: `/v3/projects/service` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:40.819974 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13222cd-93f8-40ee-9f95-d66cb2ed019e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:36:40.821390 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13222cd-93f8-40ee-9f95-d66cb2ed019e None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:36:40.832310 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-d13222cd-93f8-40ee-9f95-d66cb2ed019e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 13:36:40.832310 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:36:40.832310 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:36:40.832310 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:36:40.832310 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:36:40.832310 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:36:40.832310 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:36:40.832310 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:36:40.832310 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:36:40.832310 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:36:40.832310 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:36:40.832310 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:36:40.832310 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 13:36:40.832310 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 13:36:40.832310 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 13:36:40.833352 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 13:36:40.833352 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:36:40.833352 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:36:40.833352 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:36:40.833352 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:36:40.833352 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:36:40.833352 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:36:40.833352 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:36:40.833352 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:36:40.833352 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:36:40.833352 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:36:40.833352 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:36:40.833352 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:36:40.833352 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:36:40.833352 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:36:40.833352 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:36:40.833352 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:36:40.834402 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 13:36:40.834402 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 13:36:40.834402 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 13:36:40.834402 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 13:36:40.834402 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 13:36:40.834402 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 13:36:40.834402 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 13:36:40.834402 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:36:40.834402 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 629/1271] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Jul 26 13:36:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:36:40.839224 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c60ec0-a730-4bdd-84f2-6b27ef513171 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:36:40.896633 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:36:40.937210 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c60ec0-a730-4bdd-84f2-6b27ef513171 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:36:40.969118 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00ddd6e-509a-41b7-9da3-ec48fc3e8756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:36:40.969744 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00ddd6e-509a-41b7-9da3-ec48fc3e8756 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:40.970022 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00ddd6e-509a-41b7-9da3-ec48fc3e8756 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:40.970293 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00ddd6e-509a-41b7-9da3-ec48fc3e8756 None admin] PATH_INFO: `/v3/projects` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:40.971032 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00ddd6e-509a-41b7-9da3-ec48fc3e8756 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:36:40.971335 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00ddd6e-509a-41b7-9da3-ec48fc3e8756 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:36:40.972731 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00ddd6e-509a-41b7-9da3-ec48fc3e8756 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:36:40.984622 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 643/1272] 10.222.0.58 () {56 vars in 1148 bytes} [Tue Jul 26 13:36:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:36:40.990583 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13222cd-93f8-40ee-9f95-d66cb2ed019e None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:36:41.079957 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:36:41.088552 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13222cd-93f8-40ee-9f95-d66cb2ed019e None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:36:41.122742 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095445fc-c6a5-436f-a72f-a9501719617b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:36:41.123398 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095445fc-c6a5-436f-a72f-a9501719617b None admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:41.123678 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095445fc-c6a5-436f-a72f-a9501719617b None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:41.123952 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095445fc-c6a5-436f-a72f-a9501719617b None admin] PATH_INFO: `/v3/projects/1e8bb4385c204210a1bc77b1f0d654f3/users/835cbc13756b41b598142df3e35f5492/roles/6e0201a9a6484f7d8b112b1eafff70bd` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:41.125905 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095445fc-c6a5-436f-a72f-a9501719617b None admin] RBAC: Authorizing `identity:create_grant(project_id=1e8bb4385c204210a1bc77b1f0d654f3, user_id=835cbc13756b41b598142df3e35f5492, role_id=6e0201a9a6484f7d8b112b1eafff70bd)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:36:41.127322 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095445fc-c6a5-436f-a72f-a9501719617b None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:36:41.139196 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 630/1273] 10.222.0.58 () {58 vars in 1439 bytes} [Tue Jul 26 13:36:40 2022] PUT /identity/v3/projects/1e8bb4385c204210a1bc77b1f0d654f3/users/835cbc13756b41b598142df3e35f5492/roles/6e0201a9a6484f7d8b112b1eafff70bd => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:36:43.117068 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39fc83b-3176-4c89-991e-1f60e0c48657 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:43.117568 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39fc83b-3176-4c89-991e-1f60e0c48657 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:43.117682 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39fc83b-3176-4c89-991e-1f60e0c48657 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:43.118993 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 644/1274] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:36:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:36:43.124457 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2702b107-f6bb-4a96-b68f-7c9faa786792 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:43.124825 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2702b107-f6bb-4a96-b68f-7c9faa786792 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:43.125032 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2702b107-f6bb-4a96-b68f-7c9faa786792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:43.143992 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-2702b107-f6bb-4a96-b68f-7c9faa786792 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:36:43.145214 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-2702b107-f6bb-4a96-b68f-7c9faa786792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:36:43.209389 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 631/1275] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:36:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 13:36:43.231988 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39fc83b-3176-4c89-991e-1f60e0c48657 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:36:43.330530 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39fc83b-3176-4c89-991e-1f60e0c48657 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:36:43.347550 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:36:43.363583 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dae3b47-3074-486b-b497-36bde025501a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:36:43.364208 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dae3b47-3074-486b-b497-36bde025501a None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:43.364487 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dae3b47-3074-486b-b497-36bde025501a None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:43.364848 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dae3b47-3074-486b-b497-36bde025501a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:43.374981 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dae3b47-3074-486b-b497-36bde025501a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 13:36:43.379772 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-8dae3b47-3074-486b-b497-36bde025501a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 13:36:43.379772 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:36:43.379772 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:36:43.379772 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:36:43.379772 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:36:43.379772 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:36:43.379772 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:36:43.379772 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:36:43.379772 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:36:43.379772 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:36:43.379772 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:36:43.379772 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:36:43.379772 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 13:36:43.379772 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 13:36:43.379772 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 13:36:43.379772 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 13:36:43.380853 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 13:36:43.380853 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 13:36:43.380853 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 13:36:43.380853 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 13:36:43.380853 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:36:43.380853 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:36:43.380853 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:36:43.380853 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:36:43.380853 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:36:43.380853 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:36:43.380853 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:36:43.380853 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:36:43.380853 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:36:43.380853 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:36:43.380853 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:36:43.380853 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:36:43.381939 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:36:43.381939 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:36:43.381939 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:36:43.381939 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:36:43.381939 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 13:36:43.381939 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 13:36:43.381939 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 13:36:43.381939 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 13:36:43.381939 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 13:36:43.381939 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 13:36:43.381939 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 13:36:43.381939 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:36:43.381939 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 645/1276] 10.222.0.58 () {56 vars in 1096 bytes} [Tue Jul 26 13:36:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:36:43.386986 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2702b107-f6bb-4a96-b68f-7c9faa786792 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:36:43.466689 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:36:43.485160 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2702b107-f6bb-4a96-b68f-7c9faa786792 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:36:43.518363 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2a49dd-6785-42c2-a020-26bc75fab57c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:36:43.518978 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2a49dd-6785-42c2-a020-26bc75fab57c None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:43.519262 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2a49dd-6785-42c2-a020-26bc75fab57c None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:43.519542 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2a49dd-6785-42c2-a020-26bc75fab57c None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:43.520251 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2a49dd-6785-42c2-a020-26bc75fab57c None admin] RBAC: Adding query filter params (name=admin) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:36:43.520562 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2a49dd-6785-42c2-a020-26bc75fab57c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:36:43.521961 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2a49dd-6785-42c2-a020-26bc75fab57c None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:36:43.529823 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 632/1277] 10.222.0.58 () {56 vars in 1099 bytes} [Tue Jul 26 13:36:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:36:43.535616 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dae3b47-3074-486b-b497-36bde025501a None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:36:43.633143 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dae3b47-3074-486b-b497-36bde025501a None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:36:43.664875 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5f07c8-8dbb-4a19-b043-2dcb3650b0f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:36:43.665518 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5f07c8-8dbb-4a19-b043-2dcb3650b0f7 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:43.665802 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5f07c8-8dbb-4a19-b043-2dcb3650b0f7 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:43.666336 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5f07c8-8dbb-4a19-b043-2dcb3650b0f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:43.676949 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5f07c8-8dbb-4a19-b043-2dcb3650b0f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:36:43.678403 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5f07c8-8dbb-4a19-b043-2dcb3650b0f7 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:36:43.692641 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-1f5f07c8-8dbb-4a19-b043-2dcb3650b0f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:36:43.692641 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:36:43.692641 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:36:43.692641 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:36:43.692641 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:36:43.692641 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:36:43.692641 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:36:43.692641 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:36:43.692641 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:36:43.692641 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:36:43.692641 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:36:43.692641 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:36:43.692641 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 13:36:43.692641 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 13:36:43.692641 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 13:36:43.693642 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 13:36:43.693642 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:36:43.693642 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:36:43.693642 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:36:43.693642 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:36:43.693642 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:36:43.693642 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:36:43.693642 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:36:43.693642 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:36:43.693642 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:36:43.693642 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:36:43.693642 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:36:43.693642 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:36:43.693642 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:36:43.693642 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:36:43.693642 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:36:43.693642 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:36:43.694725 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 13:36:43.694725 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 13:36:43.694725 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:36:43.694725 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:36:43.694725 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 646/1278] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:36:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:36:43.699538 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2a49dd-6785-42c2-a020-26bc75fab57c None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:36:43.800320 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2a49dd-6785-42c2-a020-26bc75fab57c None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:36:43.832452 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdf184c-5b4b-4563-8a30-b4618f8ed162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:36:43.833063 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdf184c-5b4b-4563-8a30-b4618f8ed162 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:43.833348 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdf184c-5b4b-4563-8a30-b4618f8ed162 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:43.833655 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdf184c-5b4b-4563-8a30-b4618f8ed162 None admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:43.834342 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdf184c-5b4b-4563-8a30-b4618f8ed162 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:36:43.834648 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdf184c-5b4b-4563-8a30-b4618f8ed162 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:36:43.836006 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdf184c-5b4b-4563-8a30-b4618f8ed162 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:36:43.847894 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 633/1279] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Jul 26 13:36:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:36:43.853686 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5f07c8-8dbb-4a19-b043-2dcb3650b0f7 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:36:43.884148 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:36:43.951095 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5f07c8-8dbb-4a19-b043-2dcb3650b0f7 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:36:43.983213 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337333e5-f38e-4eee-a2d2-07ea33de604c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:36:43.984398 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337333e5-f38e-4eee-a2d2-07ea33de604c None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:43.984683 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337333e5-f38e-4eee-a2d2-07ea33de604c None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:43.984959 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337333e5-f38e-4eee-a2d2-07ea33de604c None admin] PATH_INFO: `/v3/users/placement` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:43.992283 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337333e5-f38e-4eee-a2d2-07ea33de604c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:36:43.993737 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337333e5-f38e-4eee-a2d2-07ea33de604c None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:36:44.004726 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-337333e5-f38e-4eee-a2d2-07ea33de604c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 26 13:36:44.004726 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:36:44.004726 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:36:44.004726 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:36:44.004726 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:36:44.004726 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:36:44.004726 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:36:44.004726 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:36:44.004726 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:36:44.004726 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:36:44.004726 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:36:44.004726 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:36:44.004726 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 13:36:44.004726 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 13:36:44.004726 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 13:36:44.004726 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 13:36:44.005840 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:36:44.005840 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:36:44.005840 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 13:36:44.005840 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 13:36:44.005840 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 13:36:44.005840 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 13:36:44.005840 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:36:44.005840 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:36:44.005840 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:36:44.005840 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:36:44.005840 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:36:44.005840 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:36:44.005840 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:36:44.005840 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:36:44.005840 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:36:44.005840 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:36:44.005840 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:36:44.005840 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:36:44.007182 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:36:44.007182 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:36:44.007182 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 13:36:44.007182 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 13:36:44.007182 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 13:36:44.007182 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 13:36:44.007182 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 13:36:44.007182 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 13:36:44.007182 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 26 13:36:44.007182 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:36:44.007182 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 647/1280] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:36:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:36:44.011582 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdf184c-5b4b-4563-8a30-b4618f8ed162 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:36:44.015342 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:36:44.110521 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdf184c-5b4b-4563-8a30-b4618f8ed162 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:36:44.143337 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd682c8-1510-4b19-a010-6bccef751936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:36:44.143874 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd682c8-1510-4b19-a010-6bccef751936 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:44.144157 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd682c8-1510-4b19-a010-6bccef751936 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:44.144427 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd682c8-1510-4b19-a010-6bccef751936 None admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:44.145244 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd682c8-1510-4b19-a010-6bccef751936 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:36:44.145570 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd682c8-1510-4b19-a010-6bccef751936 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:36:44.147344 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd682c8-1510-4b19-a010-6bccef751936 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:36:44.164143 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 634/1281] 10.222.0.58 () {56 vars in 1143 bytes} [Tue Jul 26 13:36:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:36:44.170079 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337333e5-f38e-4eee-a2d2-07ea33de604c None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:36:44.256707 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:36:44.267057 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337333e5-f38e-4eee-a2d2-07ea33de604c None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:36:44.299286 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3881ecac-2711-482f-894a-985a205c7a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:36:44.299888 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3881ecac-2711-482f-894a-985a205c7a19 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:44.300166 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3881ecac-2711-482f-894a-985a205c7a19 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:44.300436 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3881ecac-2711-482f-894a-985a205c7a19 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:44.364462 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3881ecac-2711-482f-894a-985a205c7a19 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:36:44.365919 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3881ecac-2711-482f-894a-985a205c7a19 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:36:44.449767 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:36:44.460197 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 648/1282] 10.222.0.58 () {58 vars in 1282 bytes} [Tue Jul 26 13:36:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 13:36:44.465886 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd682c8-1510-4b19-a010-6bccef751936 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:36:44.526264 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:36:44.565295 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd682c8-1510-4b19-a010-6bccef751936 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:36:44.598155 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42e9bbb-0998-4b2f-abb4-3cdd5654592c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:36:44.598779 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42e9bbb-0998-4b2f-abb4-3cdd5654592c None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:44.599148 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42e9bbb-0998-4b2f-abb4-3cdd5654592c None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:44.599424 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42e9bbb-0998-4b2f-abb4-3cdd5654592c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:44.609375 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42e9bbb-0998-4b2f-abb4-3cdd5654592c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:36:44.610815 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42e9bbb-0998-4b2f-abb4-3cdd5654592c None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:36:44.625663 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-f42e9bbb-0998-4b2f-abb4-3cdd5654592c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:36:44.625663 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:36:44.625663 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:36:44.625663 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:36:44.625663 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:36:44.625663 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:36:44.625663 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:36:44.625663 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:36:44.625663 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:36:44.625663 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:36:44.625663 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:36:44.625663 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:36:44.625663 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 13:36:44.625663 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 13:36:44.625663 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 13:36:44.626780 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 13:36:44.626780 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:36:44.626780 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:36:44.626780 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:36:44.626780 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:36:44.626780 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:36:44.626780 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:36:44.626780 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:36:44.626780 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:36:44.626780 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:36:44.626780 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:36:44.626780 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:36:44.626780 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:36:44.626780 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:36:44.626780 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:36:44.626780 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:36:44.626780 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:36:44.628027 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 13:36:44.628027 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 13:36:44.628027 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 13:36:44.628027 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:36:44.628027 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 635/1283] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:36:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:36:44.632617 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3881ecac-2711-482f-894a-985a205c7a19 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:36:44.728458 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3881ecac-2711-482f-894a-985a205c7a19 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:36:44.761098 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3271cd89-fcf0-4cc2-b7fe-014523012e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:36:44.761730 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3271cd89-fcf0-4cc2-b7fe-014523012e29 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:44.762011 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3271cd89-fcf0-4cc2-b7fe-014523012e29 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:44.762285 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3271cd89-fcf0-4cc2-b7fe-014523012e29 None admin] PATH_INFO: `/v3/domains` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:44.762968 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3271cd89-fcf0-4cc2-b7fe-014523012e29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:36:44.763272 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3271cd89-fcf0-4cc2-b7fe-014523012e29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:36:44.765213 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3271cd89-fcf0-4cc2-b7fe-014523012e29 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:36:44.776037 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 649/1284] 10.222.0.58 () {56 vars in 1109 bytes} [Tue Jul 26 13:36:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:36:44.781625 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42e9bbb-0998-4b2f-abb4-3cdd5654592c None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:36:44.877129 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42e9bbb-0998-4b2f-abb4-3cdd5654592c None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:36:44.890677 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:36:44.909976 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d947f2-c8b6-42f2-8e7a-375d1a68af65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:36:44.910600 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d947f2-c8b6-42f2-8e7a-375d1a68af65 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:44.910880 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d947f2-c8b6-42f2-8e7a-375d1a68af65 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:44.911155 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d947f2-c8b6-42f2-8e7a-375d1a68af65 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:44.917923 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d947f2-c8b6-42f2-8e7a-375d1a68af65 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:36:44.919348 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d947f2-c8b6-42f2-8e7a-375d1a68af65 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:36:44.930875 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-13d947f2-c8b6-42f2-8e7a-375d1a68af65 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 13:36:44.930875 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:36:44.930875 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:36:44.930875 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:36:44.930875 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:36:44.930875 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:36:44.930875 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:36:44.930875 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:36:44.930875 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:36:44.930875 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:36:44.930875 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:36:44.930875 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:36:44.930875 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 13:36:44.930875 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 13:36:44.930875 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 13:36:44.931920 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 13:36:44.931920 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:36:44.931920 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:36:44.931920 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:36:44.931920 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:36:44.931920 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:36:44.931920 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:36:44.931920 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:36:44.931920 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 13:36:44.931920 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:36:44.931920 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:36:44.931920 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:36:44.931920 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:36:44.931920 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:36:44.931920 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:36:44.931920 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:36:44.931920 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:36:44.932845 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 13:36:44.932845 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 13:36:44.932845 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 13:36:44.932845 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 13:36:44.932845 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 13:36:44.932845 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 13:36:44.932845 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 13:36:44.932845 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 13:36:44.932845 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 636/1285] 10.222.0.58 () {56 vars in 1111 bytes} [Tue Jul 26 13:36:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:36:44.938332 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3271cd89-fcf0-4cc2-b7fe-014523012e29 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:36:44.986620 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:36:45.035555 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3271cd89-fcf0-4cc2-b7fe-014523012e29 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:36:45.069901 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2574b4db-45d3-4019-aa00-8284cb03ce0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:36:45.070513 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2574b4db-45d3-4019-aa00-8284cb03ce0b None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:45.070794 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2574b4db-45d3-4019-aa00-8284cb03ce0b None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:45.071069 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2574b4db-45d3-4019-aa00-8284cb03ce0b None admin] PATH_INFO: `/v3/projects` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:45.071811 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2574b4db-45d3-4019-aa00-8284cb03ce0b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:36:45.072121 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2574b4db-45d3-4019-aa00-8284cb03ce0b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:36:45.073553 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2574b4db-45d3-4019-aa00-8284cb03ce0b None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:36:45.085116 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 650/1286] 10.222.0.58 () {56 vars in 1148 bytes} [Tue Jul 26 13:36:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:36:45.091078 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d947f2-c8b6-42f2-8e7a-375d1a68af65 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:36:45.189907 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d947f2-c8b6-42f2-8e7a-375d1a68af65 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:36:45.221744 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7649875d-d7cb-4a8e-b344-7a3e8ae0f602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:36:45.222340 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7649875d-d7cb-4a8e-b344-7a3e8ae0f602 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:45.222623 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7649875d-d7cb-4a8e-b344-7a3e8ae0f602 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:45.222896 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7649875d-d7cb-4a8e-b344-7a3e8ae0f602 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:45.223674 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7649875d-d7cb-4a8e-b344-7a3e8ae0f602 None admin] RBAC: Adding query filter params (role.id=6e0201a9a6484f7d8b112b1eafff70bd, scope.project.id=1e8bb4385c204210a1bc77b1f0d654f3, user.id=835cbc13756b41b598142df3e35f5492) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:36:45.223982 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7649875d-d7cb-4a8e-b344-7a3e8ae0f602 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:36:45.225392 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7649875d-d7cb-4a8e-b344-7a3e8ae0f602 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:36:45.231031 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 637/1287] 10.222.0.58 () {56 vars in 1374 bytes} [Tue Jul 26 13:36:45 2022] GET /identity/v3/role_assignments?user.id=835cbc13756b41b598142df3e35f5492&scope.project.id=1e8bb4385c204210a1bc77b1f0d654f3&role.id=6e0201a9a6484f7d8b112b1eafff70bd => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:36:47.257756 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bee5ced-68fd-42f8-ab8a-eb41e13c88c1 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:47.258313 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bee5ced-68fd-42f8-ab8a-eb41e13c88c1 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:47.258313 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bee5ced-68fd-42f8-ab8a-eb41e13c88c1 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:47.259651 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 651/1288] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:36:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:36:47.265203 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d17a53-1c3f-406e-bf77-359d4360e64a None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:47.265620 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d17a53-1c3f-406e-bf77-359d4360e64a None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:47.265772 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d17a53-1c3f-406e-bf77-359d4360e64a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:47.284062 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-29d17a53-1c3f-406e-bf77-359d4360e64a None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:36:47.285249 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-29d17a53-1c3f-406e-bf77-359d4360e64a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:36:47.314404 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:36:47.352617 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 638/1289] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:36:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 13:36:47.374301 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bee5ced-68fd-42f8-ab8a-eb41e13c88c1 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:36:47.470778 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bee5ced-68fd-42f8-ab8a-eb41e13c88c1 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:36:47.480849 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:36:47.502928 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d565827-2fad-4498-8141-34474e501e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:36:47.503548 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d565827-2fad-4498-8141-34474e501e29 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:47.503824 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d565827-2fad-4498-8141-34474e501e29 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:47.504092 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d565827-2fad-4498-8141-34474e501e29 None admin] PATH_INFO: `/v3/services/placement` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:47.504707 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d565827-2fad-4498-8141-34474e501e29 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:36:47.506085 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d565827-2fad-4498-8141-34474e501e29 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:36:47.515088 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-3d565827-2fad-4498-8141-34474e501e29 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 26 13:36:47.515088 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:36:47.515088 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 13:36:47.515088 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 13:36:47.515088 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 13:36:47.515088 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 13:36:47.515088 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 13:36:47.515088 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 13:36:47.515088 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 26 13:36:47.515088 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:36:47.515088 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 13:36:47.515088 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:36:47.515088 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:36:47.515088 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:36:47.515088 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:36:47.515088 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:36:47.515088 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:36:47.516299 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:36:47.516299 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:36:47.516299 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:36:47.516299 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:36:47.516299 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:36:47.516299 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:36:47.516299 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 13:36:47.516299 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 13:36:47.516299 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 13:36:47.516299 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 13:36:47.516299 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:36:47.516299 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:36:47.516299 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:36:47.516299 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:36:47.516299 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:36:47.516299 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:36:47.517573 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:36:47.517573 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:36:47.517573 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:36:47.517573 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:36:47.517573 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:36:47.517573 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:36:47.517573 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:36:47.517573 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:36:47.517573 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:36:47.517573 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:36:47.517573 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 13:36:47.517573 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 13:36:47.517573 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 26 13:36:47.517573 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:36:47.517573 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 652/1290] 10.222.0.58 () {56 vars in 1117 bytes} [Tue Jul 26 13:36:47 2022] GET /identity/v3/services/placement => generated 90 bytes in 143 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:36:47.522963 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d17a53-1c3f-406e-bf77-359d4360e64a None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:36:47.626314 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d17a53-1c3f-406e-bf77-359d4360e64a None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:36:47.659103 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6992e191-8b4a-49a2-9c04-74a04dd80178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:36:47.659713 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6992e191-8b4a-49a2-9c04-74a04dd80178 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:47.659994 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6992e191-8b4a-49a2-9c04-74a04dd80178 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:47.660268 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6992e191-8b4a-49a2-9c04-74a04dd80178 None admin] PATH_INFO: `/v3/services` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:47.660934 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6992e191-8b4a-49a2-9c04-74a04dd80178 None admin] RBAC: Adding query filter params (name=placement) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:36:47.661275 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6992e191-8b4a-49a2-9c04-74a04dd80178 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:36:47.662699 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6992e191-8b4a-49a2-9c04-74a04dd80178 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:36:47.667625 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 639/1291] 10.222.0.58 () {56 vars in 1116 bytes} [Tue Jul 26 13:36:47 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:36:47.673323 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d565827-2fad-4498-8141-34474e501e29 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:36:47.773040 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d565827-2fad-4498-8141-34474e501e29 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:36:47.812002 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fdd285-3ae4-48ed-a4ca-a76a4faef59f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:36:47.812869 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fdd285-3ae4-48ed-a4ca-a76a4faef59f None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:47.813145 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fdd285-3ae4-48ed-a4ca-a76a4faef59f None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:47.813550 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fdd285-3ae4-48ed-a4ca-a76a4faef59f None admin] PATH_INFO: `/v3/services` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:47.814636 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fdd285-3ae4-48ed-a4ca-a76a4faef59f None admin] RBAC: Adding query filter params (type=placement) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:36:47.814983 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fdd285-3ae4-48ed-a4ca-a76a4faef59f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:36:47.816732 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fdd285-3ae4-48ed-a4ca-a76a4faef59f None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:36:47.822010 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 653/1292] 10.222.0.58 () {56 vars in 1116 bytes} [Tue Jul 26 13:36:47 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:36:49.763294 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa1cdbf-c300-4c91-85fc-0975466b6ddb None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:49.763870 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa1cdbf-c300-4c91-85fc-0975466b6ddb None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:49.763870 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa1cdbf-c300-4c91-85fc-0975466b6ddb None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:49.765183 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 640/1293] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:36:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:36:49.770318 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd06aef-0f71-4551-ac8a-f4e7fa906f17 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:49.770755 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd06aef-0f71-4551-ac8a-f4e7fa906f17 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:49.770939 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd06aef-0f71-4551-ac8a-f4e7fa906f17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:49.788987 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-fcd06aef-0f71-4551-ac8a-f4e7fa906f17 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:36:49.790194 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-fcd06aef-0f71-4551-ac8a-f4e7fa906f17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:36:49.819618 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:36:49.856684 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 654/1294] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:36:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 13:36:49.879478 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa1cdbf-c300-4c91-85fc-0975466b6ddb None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:36:49.909909 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:36:49.978451 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa1cdbf-c300-4c91-85fc-0975466b6ddb None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:36:50.010622 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7860d043-49f3-4162-aba3-9f59ed84a0c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:36:50.011222 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7860d043-49f3-4162-aba3-9f59ed84a0c7 None admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:50.011499 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7860d043-49f3-4162-aba3-9f59ed84a0c7 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:50.011770 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7860d043-49f3-4162-aba3-9f59ed84a0c7 None admin] PATH_INFO: `/v3/services` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:50.012452 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7860d043-49f3-4162-aba3-9f59ed84a0c7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:36:50.013866 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7860d043-49f3-4162-aba3-9f59ed84a0c7 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:36:50.027140 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 641/1295] 10.222.0.58 () {60 vars in 1141 bytes} [Tue Jul 26 13:36:49 2022] POST /identity/v3/services => generated 246 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 13:36:51.992961 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39539ac8-3222-4667-b797-d2cc2b8d4f11 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:51.993543 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39539ac8-3222-4667-b797-d2cc2b8d4f11 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:51.993543 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39539ac8-3222-4667-b797-d2cc2b8d4f11 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:51.994920 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 655/1296] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:36:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:36:51.999880 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1771a9e9-0389-4302-9380-392e6e846371 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:52.000323 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1771a9e9-0389-4302-9380-392e6e846371 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:52.000995 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1771a9e9-0389-4302-9380-392e6e846371 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:52.018717 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-1771a9e9-0389-4302-9380-392e6e846371 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:36:52.019928 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-1771a9e9-0389-4302-9380-392e6e846371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:36:52.091664 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 642/1297] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:36:51 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 13:36:52.114569 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39539ac8-3222-4667-b797-d2cc2b8d4f11 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:36:52.212586 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39539ac8-3222-4667-b797-d2cc2b8d4f11 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:36:52.246720 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099281b9-1e79-4221-a262-c828b60686d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:36:52.247289 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099281b9-1e79-4221-a262-c828b60686d1 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:52.247578 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099281b9-1e79-4221-a262-c828b60686d1 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:52.247855 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099281b9-1e79-4221-a262-c828b60686d1 None admin] PATH_INFO: `/v3/services/placement` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:52.248514 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099281b9-1e79-4221-a262-c828b60686d1 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:36:52.249949 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099281b9-1e79-4221-a262-c828b60686d1 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:36:52.258616 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-099281b9-1e79-4221-a262-c828b60686d1 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 26 13:36:52.258616 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:36:52.258616 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 13:36:52.258616 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 13:36:52.258616 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 13:36:52.258616 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 13:36:52.258616 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 13:36:52.258616 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 13:36:52.258616 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 26 13:36:52.258616 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:36:52.258616 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 13:36:52.258616 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:36:52.258616 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:36:52.258616 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:36:52.258616 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:36:52.258616 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:36:52.258616 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:36:52.259782 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:36:52.259782 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:36:52.259782 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:36:52.259782 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:36:52.259782 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:36:52.259782 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:36:52.259782 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 13:36:52.259782 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 13:36:52.259782 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 13:36:52.259782 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 13:36:52.259782 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:36:52.259782 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:36:52.259782 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:36:52.259782 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:36:52.259782 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:36:52.259782 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:36:52.260646 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:36:52.260646 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:36:52.260646 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:36:52.260646 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:36:52.260646 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:36:52.260646 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:36:52.260646 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:36:52.260646 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:36:52.260646 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:36:52.260646 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:36:52.260646 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 13:36:52.260646 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 13:36:52.260646 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 26 13:36:52.260646 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:36:52.260646 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 656/1298] 10.222.0.58 () {56 vars in 1117 bytes} [Tue Jul 26 13:36:52 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:36:52.265772 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1771a9e9-0389-4302-9380-392e6e846371 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:36:52.284167 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:36:52.364911 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1771a9e9-0389-4302-9380-392e6e846371 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:36:52.397210 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db129f52-2c91-49d1-b9f5-25fc074522ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:36:52.397856 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db129f52-2c91-49d1-b9f5-25fc074522ab None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:52.398137 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db129f52-2c91-49d1-b9f5-25fc074522ab None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:52.398411 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db129f52-2c91-49d1-b9f5-25fc074522ab None admin] PATH_INFO: `/v3/services` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:52.399083 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db129f52-2c91-49d1-b9f5-25fc074522ab None admin] RBAC: Adding query filter params (name=placement) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:36:52.399392 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db129f52-2c91-49d1-b9f5-25fc074522ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:36:52.400754 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db129f52-2c91-49d1-b9f5-25fc074522ab None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:36:52.406263 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 643/1299] 10.222.0.58 () {56 vars in 1116 bytes} [Tue Jul 26 13:36:52 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:36:52.412161 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099281b9-1e79-4221-a262-c828b60686d1 None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:36:52.443542 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:36:52.510481 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099281b9-1e79-4221-a262-c828b60686d1 None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:36:52.543380 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4370d4e1-f35f-4c70-93ed-eda127aed887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:36:52.544033 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4370d4e1-f35f-4c70-93ed-eda127aed887 None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:52.544312 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4370d4e1-f35f-4c70-93ed-eda127aed887 None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:52.544584 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4370d4e1-f35f-4c70-93ed-eda127aed887 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:52.545327 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4370d4e1-f35f-4c70-93ed-eda127aed887 None admin] RBAC: Adding query filter params (interface=public, service_id=81e6e2d79b704e81bb271d3ac37570a1, region_id=RegionOne) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:36:52.545666 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4370d4e1-f35f-4c70-93ed-eda127aed887 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:36:52.547010 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4370d4e1-f35f-4c70-93ed-eda127aed887 None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:36:52.551858 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 657/1300] 10.222.0.58 () {56 vars in 1251 bytes} [Tue Jul 26 13:36:52 2022] GET /identity/v3/endpoints?service_id=81e6e2d79b704e81bb271d3ac37570a1&interface=public®ion_id=RegionOne => generated 194 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:36:52.557444 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db129f52-2c91-49d1-b9f5-25fc074522ab None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:36:52.636689 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:36:52.655917 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db129f52-2c91-49d1-b9f5-25fc074522ab None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:36:52.689125 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a174a084-4e20-406f-8537-937646dbff46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:36:52.689761 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a174a084-4e20-406f-8537-937646dbff46 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:52.690044 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a174a084-4e20-406f-8537-937646dbff46 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:52.690316 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a174a084-4e20-406f-8537-937646dbff46 None admin] PATH_INFO: `/v3/services` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:52.690962 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a174a084-4e20-406f-8537-937646dbff46 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:36:52.692312 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a174a084-4e20-406f-8537-937646dbff46 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:36:52.698993 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 644/1301] 10.222.0.58 () {56 vars in 1087 bytes} [Tue Jul 26 13:36:52 2022] GET /identity/v3/services => generated 2191 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:36:54.671963 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c74500-e78f-4572-a4ac-1e39b7930881 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:54.672490 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c74500-e78f-4572-a4ac-1e39b7930881 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:54.672490 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c74500-e78f-4572-a4ac-1e39b7930881 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:54.673913 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 658/1302] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:36:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:36:54.678877 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3db551-d620-434e-844c-3af3c51cfc46 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:54.679283 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3db551-d620-434e-844c-3af3c51cfc46 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:54.679438 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3db551-d620-434e-844c-3af3c51cfc46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:54.697683 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-ae3db551-d620-434e-844c-3af3c51cfc46 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:36:54.698865 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-ae3db551-d620-434e-844c-3af3c51cfc46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:36:54.767740 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 645/1303] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:36:54 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 13:36:54.792183 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c74500-e78f-4572-a4ac-1e39b7930881 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:36:54.888029 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:36:54.895013 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c74500-e78f-4572-a4ac-1e39b7930881 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:36:54.927409 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab36dc4-aff5-4d02-a2f2-6f5c9684109f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:36:54.927983 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab36dc4-aff5-4d02-a2f2-6f5c9684109f None admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:54.928263 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab36dc4-aff5-4d02-a2f2-6f5c9684109f None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:54.928536 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab36dc4-aff5-4d02-a2f2-6f5c9684109f None admin] PATH_INFO: `/v3/services/placement` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:54.929156 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab36dc4-aff5-4d02-a2f2-6f5c9684109f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:36:54.930538 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab36dc4-aff5-4d02-a2f2-6f5c9684109f None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:36:54.939833 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application [None req-4ab36dc4-aff5-4d02-a2f2-6f5c9684109f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 26 13:36:54.939833 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:36:54.939833 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 13:36:54.939833 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 13:36:54.939833 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 13:36:54.939833 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 13:36:54.939833 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 13:36:54.939833 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 13:36:54.939833 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 26 13:36:54.939833 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:36:54.939833 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 13:36:54.939833 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:36:54.939833 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 13:36:54.939833 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 13:36:54.939833 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 13:36:54.939833 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 13:36:54.939833 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 13:36:54.940957 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 13:36:54.940957 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 13:36:54.940957 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 13:36:54.940957 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 13:36:54.940957 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 13:36:54.940957 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 13:36:54.940957 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 13:36:54.940957 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 13:36:54.940957 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 13:36:54.940957 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 13:36:54.940957 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 13:36:54.940957 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 13:36:54.940957 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 13:36:54.940957 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 13:36:54.940957 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 13:36:54.940957 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 13:36:54.942278 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 13:36:54.942278 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application with Lock( Jul 26 13:36:54.942278 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 13:36:54.942278 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self._enter() Jul 26 13:36:54.942278 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 13:36:54.942278 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 13:36:54.942278 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 13:36:54.942278 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application return self.creator() Jul 26 13:36:54.942278 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 13:36:54.942278 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application created_value = creator( Jul 26 13:36:54.942278 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 13:36:54.942278 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 13:36:54.942278 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 26 13:36:54.942278 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: ERROR keystone.server.flask.application Jul 26 13:36:54.942278 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 659/1304] 10.222.0.58 () {56 vars in 1117 bytes} [Tue Jul 26 13:36:54 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 13:36:54.947019 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3db551-d620-434e-844c-3af3c51cfc46 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:36:55.020260 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:36:55.046687 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3db551-d620-434e-844c-3af3c51cfc46 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:36:55.078787 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1bf4f9-3abf-411c-bd89-b88f9f16d77d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:36:55.079400 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1bf4f9-3abf-411c-bd89-b88f9f16d77d None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:55.079678 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1bf4f9-3abf-411c-bd89-b88f9f16d77d None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:55.079947 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1bf4f9-3abf-411c-bd89-b88f9f16d77d None admin] PATH_INFO: `/v3/services` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:55.080612 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1bf4f9-3abf-411c-bd89-b88f9f16d77d None admin] RBAC: Adding query filter params (name=placement) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:36:55.080914 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1bf4f9-3abf-411c-bd89-b88f9f16d77d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:36:55.082283 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1bf4f9-3abf-411c-bd89-b88f9f16d77d None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:36:55.087192 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 646/1305] 10.222.0.58 () {56 vars in 1116 bytes} [Tue Jul 26 13:36:54 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:36:55.093580 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab36dc4-aff5-4d02-a2f2-6f5c9684109f None admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:36:55.191105 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab36dc4-aff5-4d02-a2f2-6f5c9684109f None admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:36:55.223816 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76550c53-8e94-41ea-b422-56cf3840f22d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:36:55.224419 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76550c53-8e94-41ea-b422-56cf3840f22d None admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:36:55.224701 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76550c53-8e94-41ea-b422-56cf3840f22d None admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:36:55.224976 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76550c53-8e94-41ea-b422-56cf3840f22d None admin] PATH_INFO: `/v3/endpoints` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:36:55.225713 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76550c53-8e94-41ea-b422-56cf3840f22d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:36:55.227062 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76550c53-8e94-41ea-b422-56cf3840f22d None admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:36:55.246938 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 660/1306] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 13:36:55 2022] POST /identity/v3/endpoints => generated 331 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 13:37:05.084291 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=30 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:37:05.242452 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:37:57.279234 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12b7ef6-42d1-425d-99a2-2d423d621347 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:37:57.279720 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12b7ef6-42d1-425d-99a2-2d423d621347 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:37:57.279823 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12b7ef6-42d1-425d-99a2-2d423d621347 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:37:57.281417 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 647/1307] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:37:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:37:57.286693 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ade8b5-e4e1-44ba-a267-db9717184aff None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:37:57.287120 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ade8b5-e4e1-44ba-a267-db9717184aff None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:37:57.287246 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ade8b5-e4e1-44ba-a267-db9717184aff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:37:57.364689 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-69ade8b5-e4e1-44ba-a267-db9717184aff None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:37:57.365906 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-69ade8b5-e4e1-44ba-a267-db9717184aff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:37:57.486003 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 661/1308] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:37:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 13:37:57.530055 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22447073-0396-47db-b71c-fefb26533738 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:37:57.530259 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22447073-0396-47db-b71c-fefb26533738 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:37:57.531019 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22447073-0396-47db-b71c-fefb26533738 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:37:57.534203 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 648/1309] 10.222.0.58 () {54 vars in 965 bytes} [Tue Jul 26 13:37:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:37:57.541431 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0804240-7690-48a3-933d-3e7945f1d821 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:37:57.541779 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0804240-7690-48a3-933d-3e7945f1d821 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:37:57.542593 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0804240-7690-48a3-933d-3e7945f1d821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:37:57.661077 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-f0804240-7690-48a3-933d-3e7945f1d821 None None] MFA Rules not processed for user `3223a78866d94c2ba66402911c2dfae9`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:37:57.662354 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-f0804240-7690-48a3-933d-3e7945f1d821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:37:57.687059 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:37:57.747051 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 662/1310] 10.222.0.58 () {58 vars in 1062 bytes} [Tue Jul 26 13:37:57 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:37:57.756052 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22447073-0396-47db-b71c-fefb26533738 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:37:57.757978 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-22447073-0396-47db-b71c-fefb26533738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:37:57.884286 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22447073-0396-47db-b71c-fefb26533738 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:37:57.962742 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8392a7-a73d-4da2-b3dd-a753858453cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:37:57.963361 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8392a7-a73d-4da2-b3dd-a753858453cf service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:37:57.963643 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8392a7-a73d-4da2-b3dd-a753858453cf service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:37:57.964487 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8392a7-a73d-4da2-b3dd-a753858453cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:37:57.966178 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-6d8392a7-a73d-4da2-b3dd-a753858453cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:37:58.026831 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:37:58.089337 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8392a7-a73d-4da2-b3dd-a753858453cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:37:58.090929 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8392a7-a73d-4da2-b3dd-a753858453cf service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:37:58.111801 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 649/1311] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:37:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:38:00.376354 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18243193-5596-4272-a8da-424fc67231b2 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:38:00.376918 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18243193-5596-4272-a8da-424fc67231b2 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:38:00.376918 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18243193-5596-4272-a8da-424fc67231b2 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:38:00.378439 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 663/1312] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:38:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:38:00.383506 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a39957a-fdbc-42fe-aa63-444d930edf18 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:38:00.383897 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a39957a-fdbc-42fe-aa63-444d930edf18 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:38:00.384163 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a39957a-fdbc-42fe-aa63-444d930edf18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:38:00.464134 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-6a39957a-fdbc-42fe-aa63-444d930edf18 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:38:00.465339 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-6a39957a-fdbc-42fe-aa63-444d930edf18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:38:00.561995 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 650/1313] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:38:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:38:00.600687 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18243193-5596-4272-a8da-424fc67231b2 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:38:00.602273 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-18243193-5596-4272-a8da-424fc67231b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:38:00.712558 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18243193-5596-4272-a8da-424fc67231b2 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:38:00.782583 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:38:00.785195 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d24068-c1fd-4237-82d8-11945fe79d25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:38:00.785983 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d24068-c1fd-4237-82d8-11945fe79d25 service neutron] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:38:00.786256 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d24068-c1fd-4237-82d8-11945fe79d25 service neutron] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:38:00.786554 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d24068-c1fd-4237-82d8-11945fe79d25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:38:00.789017 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-43d24068-c1fd-4237-82d8-11945fe79d25 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:38:00.900360 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d24068-c1fd-4237-82d8-11945fe79d25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:38:00.902338 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d24068-c1fd-4237-82d8-11945fe79d25 service neutron] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:38:00.920433 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 664/1314] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:38:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:38:03.203806 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a097a4f-3c66-452e-9c41-3979a2605096 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:38:03.204380 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a097a4f-3c66-452e-9c41-3979a2605096 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:38:03.204380 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a097a4f-3c66-452e-9c41-3979a2605096 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:38:03.205743 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 651/1315] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:38:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:38:03.211716 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67274e2-2352-4d06-84b0-a2bc7408d221 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:38:03.212192 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67274e2-2352-4d06-84b0-a2bc7408d221 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:38:03.212381 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67274e2-2352-4d06-84b0-a2bc7408d221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:38:03.278681 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-d67274e2-2352-4d06-84b0-a2bc7408d221 None None] MFA Rules not processed for user `eac6099386504fcea65a5d57e8a6c1d2`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:38:03.280122 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-d67274e2-2352-4d06-84b0-a2bc7408d221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:38:03.331225 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:38:03.375402 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 665/1316] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:38:03 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 13:38:03.415043 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a097a4f-3c66-452e-9c41-3979a2605096 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:38:03.416613 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-2a097a4f-3c66-452e-9c41-3979a2605096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:38:03.426582 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:38:03.524536 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a097a4f-3c66-452e-9c41-3979a2605096 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:38:03.596840 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542bea79-ac10-40c2-a06d-781c60c4406c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:38:03.597458 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542bea79-ac10-40c2-a06d-781c60c4406c service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:38:03.597761 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542bea79-ac10-40c2-a06d-781c60c4406c service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:38:03.598030 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542bea79-ac10-40c2-a06d-781c60c4406c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:38:03.599628 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-542bea79-ac10-40c2-a06d-781c60c4406c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:38:03.662809 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:38:03.711799 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542bea79-ac10-40c2-a06d-781c60c4406c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:38:03.713236 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542bea79-ac10-40c2-a06d-781c60c4406c service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:38:03.731871 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 652/1317] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:38:03 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:38:06.280324 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29590720-7ec8-4f7c-8c40-d4b40b82c1d4 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:38:06.280884 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29590720-7ec8-4f7c-8c40-d4b40b82c1d4 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:38:06.280884 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29590720-7ec8-4f7c-8c40-d4b40b82c1d4 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:38:06.282273 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 666/1318] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:38:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:38:06.287583 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cff316-07f3-4bc2-bfbc-51fe37574122 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:38:06.287953 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cff316-07f3-4bc2-bfbc-51fe37574122 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:38:06.288430 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cff316-07f3-4bc2-bfbc-51fe37574122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:38:06.352985 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-e1cff316-07f3-4bc2-bfbc-51fe37574122 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:38:06.354227 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-e1cff316-07f3-4bc2-bfbc-51fe37574122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:38:06.451300 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 653/1319] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:38:06 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:38:06.489269 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29590720-7ec8-4f7c-8c40-d4b40b82c1d4 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:38:06.490819 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-29590720-7ec8-4f7c-8c40-d4b40b82c1d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:38:06.591341 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29590720-7ec8-4f7c-8c40-d4b40b82c1d4 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:38:06.661269 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c4f378-2ee9-4119-a262-1b6e8a4f00da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:38:06.661842 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c4f378-2ee9-4119-a262-1b6e8a4f00da service neutron] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:38:06.662120 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c4f378-2ee9-4119-a262-1b6e8a4f00da service neutron] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:38:06.662391 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c4f378-2ee9-4119-a262-1b6e8a4f00da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:38:06.664238 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-76c4f378-2ee9-4119-a262-1b6e8a4f00da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:38:06.671941 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:38:06.772758 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c4f378-2ee9-4119-a262-1b6e8a4f00da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:38:06.774218 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c4f378-2ee9-4119-a262-1b6e8a4f00da service neutron] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:38:06.792957 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 667/1320] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:38:06 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:38:09.420080 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4492818c-1364-4d8e-bf53-41b61c8d2afc None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:38:09.420644 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4492818c-1364-4d8e-bf53-41b61c8d2afc None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:38:09.420644 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4492818c-1364-4d8e-bf53-41b61c8d2afc None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:38:09.422044 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 654/1321] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:38:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:38:09.427024 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197d651a-f3e6-41dd-8afc-ea2d0c0b2ac3 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:38:09.427424 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197d651a-f3e6-41dd-8afc-ea2d0c0b2ac3 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:38:09.427576 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197d651a-f3e6-41dd-8afc-ea2d0c0b2ac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:38:09.492891 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-197d651a-f3e6-41dd-8afc-ea2d0c0b2ac3 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:38:09.494230 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-197d651a-f3e6-41dd-8afc-ea2d0c0b2ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:38:09.553109 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:38:09.596228 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 668/1322] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:38:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:38:09.648793 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22f108b-41ce-4ff5-9ef5-cf1f76ff442a None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:38:09.649074 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22f108b-41ce-4ff5-9ef5-cf1f76ff442a None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:38:09.649352 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22f108b-41ce-4ff5-9ef5-cf1f76ff442a None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:38:09.650796 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 655/1323] 10.222.0.58 () {54 vars in 965 bytes} [Tue Jul 26 13:38:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:38:09.657281 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b01b1f-4c16-42e3-b611-ef4709f8e651 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:38:09.657694 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b01b1f-4c16-42e3-b611-ef4709f8e651 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:38:09.658040 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b01b1f-4c16-42e3-b611-ef4709f8e651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:38:09.739333 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-08b01b1f-4c16-42e3-b611-ef4709f8e651 None None] MFA Rules not processed for user `3223a78866d94c2ba66402911c2dfae9`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:38:09.740526 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-08b01b1f-4c16-42e3-b611-ef4709f8e651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:38:09.819458 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 669/1324] 10.222.0.58 () {58 vars in 1062 bytes} [Tue Jul 26 13:38:09 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:38:09.828519 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22f108b-41ce-4ff5-9ef5-cf1f76ff442a None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:38:09.830165 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-b22f108b-41ce-4ff5-9ef5-cf1f76ff442a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:38:09.849499 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:38:09.934135 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22f108b-41ce-4ff5-9ef5-cf1f76ff442a None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:38:10.006029 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c8362b-c08c-43ad-8c53-00851b944779 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:38:10.006542 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c8362b-c08c-43ad-8c53-00851b944779 service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:38:10.006822 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c8362b-c08c-43ad-8c53-00851b944779 service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:38:10.007097 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c8362b-c08c-43ad-8c53-00851b944779 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:38:10.008716 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-89c8362b-c08c-43ad-8c53-00851b944779 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:38:10.082042 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:38:10.122843 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c8362b-c08c-43ad-8c53-00851b944779 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:38:10.124309 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c8362b-c08c-43ad-8c53-00851b944779 service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:38:10.143712 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 656/1325] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:38:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:38:12.411215 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aa6edc-16ec-4587-80c3-dbbe2ef88b29 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:38:12.411732 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aa6edc-16ec-4587-80c3-dbbe2ef88b29 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:38:12.411732 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aa6edc-16ec-4587-80c3-dbbe2ef88b29 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:38:12.413374 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 670/1326] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:38:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:38:12.418428 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f3ada6-712e-4f2a-ba59-026837087521 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:38:12.418816 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f3ada6-712e-4f2a-ba59-026837087521 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:38:12.419014 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f3ada6-712e-4f2a-ba59-026837087521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:38:12.489351 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-c1f3ada6-712e-4f2a-ba59-026837087521 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:38:12.490590 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-c1f3ada6-712e-4f2a-ba59-026837087521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:38:12.586269 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 657/1327] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:38:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:38:12.622379 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aa6edc-16ec-4587-80c3-dbbe2ef88b29 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:38:12.623922 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-c2aa6edc-16ec-4587-80c3-dbbe2ef88b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:38:12.693853 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:38:12.737810 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aa6edc-16ec-4587-80c3-dbbe2ef88b29 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:38:12.812429 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67902ebd-f341-449c-9a45-fc420a4c8971 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:38:12.813049 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67902ebd-f341-449c-9a45-fc420a4c8971 service neutron] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:38:12.813334 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67902ebd-f341-449c-9a45-fc420a4c8971 service neutron] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:38:12.813652 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67902ebd-f341-449c-9a45-fc420a4c8971 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:38:12.815278 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-67902ebd-f341-449c-9a45-fc420a4c8971 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:38:12.923538 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:38:12.925889 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67902ebd-f341-449c-9a45-fc420a4c8971 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:38:12.927622 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67902ebd-f341-449c-9a45-fc420a4c8971 service neutron] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:38:12.946977 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 671/1328] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:38:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 13:38:14.985372 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493340c8-4a01-41d7-916f-3f52cf919e64 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:38:14.985856 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493340c8-4a01-41d7-916f-3f52cf919e64 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:38:14.985945 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493340c8-4a01-41d7-916f-3f52cf919e64 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:38:14.987299 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 658/1329] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:38:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:38:14.992376 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3762811-49f2-4d2b-b6d3-7f98b0d038fb None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:38:14.992824 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3762811-49f2-4d2b-b6d3-7f98b0d038fb None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:38:14.993050 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3762811-49f2-4d2b-b6d3-7f98b0d038fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:38:15.065032 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-c3762811-49f2-4d2b-b6d3-7f98b0d038fb None None] MFA Rules not processed for user `eac6099386504fcea65a5d57e8a6c1d2`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:38:15.066306 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-c3762811-49f2-4d2b-b6d3-7f98b0d038fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:38:15.162362 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 672/1330] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:38:14 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:38:15.200448 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493340c8-4a01-41d7-916f-3f52cf919e64 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:38:15.202039 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-493340c8-4a01-41d7-916f-3f52cf919e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:38:15.246849 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:38:15.329304 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493340c8-4a01-41d7-916f-3f52cf919e64 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:38:15.402226 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75296274-68c1-43f2-a79d-1e9d13d7452e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:38:15.402849 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75296274-68c1-43f2-a79d-1e9d13d7452e service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:38:15.403126 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75296274-68c1-43f2-a79d-1e9d13d7452e service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:38:15.403397 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75296274-68c1-43f2-a79d-1e9d13d7452e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:38:15.405006 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-75296274-68c1-43f2-a79d-1e9d13d7452e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:38:15.481715 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:38:15.513615 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75296274-68c1-43f2-a79d-1e9d13d7452e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:38:15.515054 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75296274-68c1-43f2-a79d-1e9d13d7452e service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:38:15.533366 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 659/1331] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:38:15 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:38:17.955509 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e133f0d-be3a-4e18-8466-5f99fdd71d50 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:38:17.956107 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e133f0d-be3a-4e18-8466-5f99fdd71d50 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:38:17.956107 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e133f0d-be3a-4e18-8466-5f99fdd71d50 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:38:17.957413 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 673/1332] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:38:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:38:17.962603 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21f6561-5601-434d-bfdc-7e576b890cb3 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:38:17.963574 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21f6561-5601-434d-bfdc-7e576b890cb3 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:38:17.963731 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21f6561-5601-434d-bfdc-7e576b890cb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:38:18.028116 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-b21f6561-5601-434d-bfdc-7e576b890cb3 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:38:18.029300 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-b21f6561-5601-434d-bfdc-7e576b890cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:38:18.124265 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 660/1333] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:38:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:38:18.162144 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e133f0d-be3a-4e18-8466-5f99fdd71d50 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:38:18.163969 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-9e133f0d-be3a-4e18-8466-5f99fdd71d50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:38:18.231684 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:38:18.265491 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e133f0d-be3a-4e18-8466-5f99fdd71d50 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:38:18.336941 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8561e80b-44bb-499a-a86b-e2b7d180a14f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:38:18.337605 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8561e80b-44bb-499a-a86b-e2b7d180a14f service neutron] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:38:18.338408 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8561e80b-44bb-499a-a86b-e2b7d180a14f service neutron] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:38:18.338682 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8561e80b-44bb-499a-a86b-e2b7d180a14f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:38:18.340296 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-8561e80b-44bb-499a-a86b-e2b7d180a14f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:38:18.444814 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8561e80b-44bb-499a-a86b-e2b7d180a14f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:38:18.446259 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8561e80b-44bb-499a-a86b-e2b7d180a14f service neutron] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:38:18.455962 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:38:18.464526 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 674/1334] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:38:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:38:20.485902 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97a2f19-44eb-4167-acc5-eb0685ae6f8d None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:38:20.486491 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97a2f19-44eb-4167-acc5-eb0685ae6f8d None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:38:20.486491 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97a2f19-44eb-4167-acc5-eb0685ae6f8d None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:38:20.487831 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 661/1335] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:38:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:38:20.492930 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b880e65-c9a6-4192-bfa7-20dc9c6ae3f0 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:38:20.493336 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b880e65-c9a6-4192-bfa7-20dc9c6ae3f0 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:38:20.493524 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b880e65-c9a6-4192-bfa7-20dc9c6ae3f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:38:20.559151 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-5b880e65-c9a6-4192-bfa7-20dc9c6ae3f0 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:38:20.560356 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-5b880e65-c9a6-4192-bfa7-20dc9c6ae3f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:38:20.654859 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 675/1336] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:38:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:38:20.695342 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97a2f19-44eb-4167-acc5-eb0685ae6f8d None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:38:20.696869 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-c97a2f19-44eb-4167-acc5-eb0685ae6f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:38:20.736825 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:38:20.799980 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97a2f19-44eb-4167-acc5-eb0685ae6f8d None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:38:20.871260 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadb8bce-597a-4281-93a1-f15b10c04c50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:38:20.871873 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadb8bce-597a-4281-93a1-f15b10c04c50 service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:38:20.872152 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadb8bce-597a-4281-93a1-f15b10c04c50 service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:38:20.872423 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadb8bce-597a-4281-93a1-f15b10c04c50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:38:20.874048 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-fadb8bce-597a-4281-93a1-f15b10c04c50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:38:20.955802 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:38:20.981927 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadb8bce-597a-4281-93a1-f15b10c04c50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:38:20.983350 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadb8bce-597a-4281-93a1-f15b10c04c50 service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:38:21.000963 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 662/1337] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:38:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:38:23.431746 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140c3e48-f9a0-4966-9880-ad1129daad17 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:38:23.432319 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140c3e48-f9a0-4966-9880-ad1129daad17 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:38:23.432319 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140c3e48-f9a0-4966-9880-ad1129daad17 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:38:23.433653 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 676/1338] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:38:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:38:23.438753 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fcd407-0ded-43cb-bef7-96b6531416c4 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:38:23.439201 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fcd407-0ded-43cb-bef7-96b6531416c4 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:38:23.439296 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fcd407-0ded-43cb-bef7-96b6531416c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:38:23.504525 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-20fcd407-0ded-43cb-bef7-96b6531416c4 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:38:23.505742 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-20fcd407-0ded-43cb-bef7-96b6531416c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:38:23.602645 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 663/1339] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:38:23 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:38:23.637963 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140c3e48-f9a0-4966-9880-ad1129daad17 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:38:23.639509 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-140c3e48-f9a0-4966-9880-ad1129daad17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:38:23.714907 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:38:23.742772 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140c3e48-f9a0-4966-9880-ad1129daad17 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:38:23.820955 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c392bf59-bea8-402c-8046-615e08fbb673 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:38:23.821606 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c392bf59-bea8-402c-8046-615e08fbb673 service neutron] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:38:23.821916 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c392bf59-bea8-402c-8046-615e08fbb673 service neutron] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:38:23.822193 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c392bf59-bea8-402c-8046-615e08fbb673 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:38:23.823819 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-c392bf59-bea8-402c-8046-615e08fbb673 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:38:23.930812 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c392bf59-bea8-402c-8046-615e08fbb673 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:38:23.932274 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c392bf59-bea8-402c-8046-615e08fbb673 service neutron] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:38:23.950509 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 677/1340] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:38:23 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:38:27.913025 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e93982-8919-4470-a356-557f51cade80 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:38:27.913763 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e93982-8919-4470-a356-557f51cade80 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:38:27.913763 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e93982-8919-4470-a356-557f51cade80 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:38:27.914974 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 664/1341] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:38:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:38:27.920613 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be40a4d0-a617-4e9a-a23f-6162fc0cfe2f None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:38:27.921022 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be40a4d0-a617-4e9a-a23f-6162fc0cfe2f None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:38:27.921193 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be40a4d0-a617-4e9a-a23f-6162fc0cfe2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:38:27.926871 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:38:27.988904 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-be40a4d0-a617-4e9a-a23f-6162fc0cfe2f None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:38:27.990130 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-be40a4d0-a617-4e9a-a23f-6162fc0cfe2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:38:28.088131 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 678/1342] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:38:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:38:28.126613 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e93982-8919-4470-a356-557f51cade80 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:38:28.128240 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-c0e93982-8919-4470-a356-557f51cade80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:38:28.183419 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:38:28.240892 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e93982-8919-4470-a356-557f51cade80 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:38:28.314359 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ae4a9c-2468-4c1f-80c1-e2fce7c74f5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:38:28.314892 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ae4a9c-2468-4c1f-80c1-e2fce7c74f5e service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:38:28.315195 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ae4a9c-2468-4c1f-80c1-e2fce7c74f5e service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:38:28.315466 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ae4a9c-2468-4c1f-80c1-e2fce7c74f5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:38:28.317068 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-d5ae4a9c-2468-4c1f-80c1-e2fce7c74f5e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:38:28.410757 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:38:28.428013 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ae4a9c-2468-4c1f-80c1-e2fce7c74f5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:38:28.429453 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ae4a9c-2468-4c1f-80c1-e2fce7c74f5e service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:38:28.448011 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 665/1343] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:38:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:38:31.086304 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea40490-88ab-4b8b-983e-dcaabf11bcfe None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:38:31.086865 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea40490-88ab-4b8b-983e-dcaabf11bcfe None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:38:31.086865 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea40490-88ab-4b8b-983e-dcaabf11bcfe None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:38:31.088395 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 679/1344] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:38:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:38:31.094191 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24313ec6-4762-4d62-a08e-fcc211f88d6a None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:38:31.094700 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24313ec6-4762-4d62-a08e-fcc211f88d6a None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:38:31.094780 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24313ec6-4762-4d62-a08e-fcc211f88d6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:38:31.167565 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-24313ec6-4762-4d62-a08e-fcc211f88d6a None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:38:31.168798 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-24313ec6-4762-4d62-a08e-fcc211f88d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:38:31.269761 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 666/1345] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:38:31 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:38:31.306799 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea40490-88ab-4b8b-983e-dcaabf11bcfe None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:38:31.308317 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-2ea40490-88ab-4b8b-983e-dcaabf11bcfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:38:31.395002 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:38:31.411909 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea40490-88ab-4b8b-983e-dcaabf11bcfe None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:38:31.483830 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7a6d8b-0e4d-4aad-b788-276318d93dc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:38:31.484369 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7a6d8b-0e4d-4aad-b788-276318d93dc7 service neutron] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:38:31.484650 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7a6d8b-0e4d-4aad-b788-276318d93dc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:38:31.484921 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7a6d8b-0e4d-4aad-b788-276318d93dc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:38:31.486573 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-bc7a6d8b-0e4d-4aad-b788-276318d93dc7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:38:31.594531 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7a6d8b-0e4d-4aad-b788-276318d93dc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:38:31.595957 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7a6d8b-0e4d-4aad-b788-276318d93dc7 service neutron] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:38:31.614250 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 680/1346] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:38:31 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:38:35.849681 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561f1fb5-c4a5-4ee8-93f6-b9b353f80936 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:38:35.850258 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561f1fb5-c4a5-4ee8-93f6-b9b353f80936 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:38:35.850354 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561f1fb5-c4a5-4ee8-93f6-b9b353f80936 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:38:35.851804 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 667/1347] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:38:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:38:35.857336 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9fd783-a4d3-4953-b58a-2205fc5de7a7 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:38:35.857797 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9fd783-a4d3-4953-b58a-2205fc5de7a7 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:38:35.857983 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9fd783-a4d3-4953-b58a-2205fc5de7a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:38:35.874330 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:38:35.932143 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-ca9fd783-a4d3-4953-b58a-2205fc5de7a7 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:38:35.933359 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-ca9fd783-a4d3-4953-b58a-2205fc5de7a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:38:36.034338 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 681/1348] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:38:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:38:36.072433 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561f1fb5-c4a5-4ee8-93f6-b9b353f80936 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:38:36.074070 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-561f1fb5-c4a5-4ee8-93f6-b9b353f80936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:38:36.136457 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:38:36.186400 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561f1fb5-c4a5-4ee8-93f6-b9b353f80936 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:38:36.262627 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4330ce7-a0c7-4fec-9e76-3f3f202cf3da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:38:36.263255 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4330ce7-a0c7-4fec-9e76-3f3f202cf3da service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:38:36.263536 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4330ce7-a0c7-4fec-9e76-3f3f202cf3da service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:38:36.263810 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4330ce7-a0c7-4fec-9e76-3f3f202cf3da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:38:36.265617 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-c4330ce7-a0c7-4fec-9e76-3f3f202cf3da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:38:36.378193 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:38:36.382552 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4330ce7-a0c7-4fec-9e76-3f3f202cf3da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:38:36.384105 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4330ce7-a0c7-4fec-9e76-3f3f202cf3da service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:38:36.402585 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 668/1349] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:38:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:38:41.322955 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947f946c-17ff-46ea-b164-144104cebe17 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:38:41.322955 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947f946c-17ff-46ea-b164-144104cebe17 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:38:41.322955 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947f946c-17ff-46ea-b164-144104cebe17 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:38:41.323793 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 682/1350] 10.222.0.58 () {54 vars in 921 bytes} [Tue Jul 26 13:38:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:38:41.330074 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2920e3b5-8a22-4686-b8b4-8b77550b0818 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:38:41.331024 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2920e3b5-8a22-4686-b8b4-8b77550b0818 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:38:41.331504 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2920e3b5-8a22-4686-b8b4-8b77550b0818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:38:41.433351 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-2920e3b5-8a22-4686-b8b4-8b77550b0818 None None] MFA Rules not processed for user `835cbc13756b41b598142df3e35f5492`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:38:41.434813 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-2920e3b5-8a22-4686-b8b4-8b77550b0818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:38:41.564564 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 669/1351] 10.222.0.58 () {58 vars in 1018 bytes} [Tue Jul 26 13:38:41 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:38:45.709942 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8021b210-0311-43fd-a6d3-cb8fbf8f43e1 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:38:45.710488 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8021b210-0311-43fd-a6d3-cb8fbf8f43e1 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:38:45.710488 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8021b210-0311-43fd-a6d3-cb8fbf8f43e1 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:38:45.711934 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 683/1352] 10.222.0.58 () {54 vars in 921 bytes} [Tue Jul 26 13:38:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:38:45.717265 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb0990e-e7dc-4f36-850d-e4d149c85cac None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:38:45.717731 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb0990e-e7dc-4f36-850d-e4d149c85cac None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:38:45.717877 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb0990e-e7dc-4f36-850d-e4d149c85cac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:38:45.780807 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:38:45.810112 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-7fb0990e-e7dc-4f36-850d-e4d149c85cac None None] MFA Rules not processed for user `835cbc13756b41b598142df3e35f5492`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:38:45.811344 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-7fb0990e-e7dc-4f36-850d-e4d149c85cac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:38:45.928399 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 670/1353] 10.222.0.58 () {58 vars in 1018 bytes} [Tue Jul 26 13:38:45 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:38:46.034342 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=59 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:38:46.975378 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263e6363-be3a-4b29-b8b3-3a064d689118 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:38:46.976058 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263e6363-be3a-4b29-b8b3-3a064d689118 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:38:46.976058 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263e6363-be3a-4b29-b8b3-3a064d689118 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:38:46.977306 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 684/1354] 10.222.0.58 () {54 vars in 921 bytes} [Tue Jul 26 13:38:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:38:46.982678 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b238fda-fded-4c13-a903-011cd08e5671 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:38:46.982678 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b238fda-fded-4c13-a903-011cd08e5671 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:38:46.983101 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b238fda-fded-4c13-a903-011cd08e5671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:38:47.062449 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-1b238fda-fded-4c13-a903-011cd08e5671 None None] MFA Rules not processed for user `835cbc13756b41b598142df3e35f5492`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:38:47.063637 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-1b238fda-fded-4c13-a903-011cd08e5671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:38:47.105724 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:38:47.170370 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 671/1355] 10.222.0.58 () {58 vars in 1018 bytes} [Tue Jul 26 13:38:46 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 13:38:50.676907 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c252de-0c23-422b-8ea1-d36bda6a01b1 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:38:50.677962 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c252de-0c23-422b-8ea1-d36bda6a01b1 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:38:50.678446 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c252de-0c23-422b-8ea1-d36bda6a01b1 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:38:50.681346 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 685/1356] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:38:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 13:38:50.686977 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6860ccc0-1d8e-404b-9587-7ceec6b2beab None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:38:50.687908 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6860ccc0-1d8e-404b-9587-7ceec6b2beab None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:38:50.688243 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6860ccc0-1d8e-404b-9587-7ceec6b2beab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:38:50.769807 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-6860ccc0-1d8e-404b-9587-7ceec6b2beab None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:38:50.771008 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-6860ccc0-1d8e-404b-9587-7ceec6b2beab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:38:50.873300 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 672/1357] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:38:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:38:50.928100 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6d5b82-8e6b-4aed-8662-80a0373e8f4d None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:38:50.928911 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6d5b82-8e6b-4aed-8662-80a0373e8f4d None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:38:50.929384 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6d5b82-8e6b-4aed-8662-80a0373e8f4d None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:38:50.931595 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 686/1358] 10.222.0.58 () {54 vars in 962 bytes} [Tue Jul 26 13:38:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:38:50.940643 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d37bfe-5da5-4064-bd79-57b2be77784f None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:38:50.941337 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d37bfe-5da5-4064-bd79-57b2be77784f None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:38:50.941833 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d37bfe-5da5-4064-bd79-57b2be77784f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:38:50.960682 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:38:51.028973 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-44d37bfe-5da5-4064-bd79-57b2be77784f None None] MFA Rules not processed for user `c69161cd520b4b29bf50f3bf2f1c033b`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:38:51.030493 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-44d37bfe-5da5-4064-bd79-57b2be77784f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:38:51.159207 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 673/1359] 10.222.0.58 () {58 vars in 1059 bytes} [Tue Jul 26 13:38:50 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:38:51.168000 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6d5b82-8e6b-4aed-8662-80a0373e8f4d None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:38:51.169530 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-6f6d5b82-8e6b-4aed-8662-80a0373e8f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:38:51.318396 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6d5b82-8e6b-4aed-8662-80a0373e8f4d None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:38:51.393324 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:38:51.441046 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f2d888-5d6f-4b80-8a00-4f87d0487541 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c69161cd520b4b29bf50f3bf2f1c033b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:38:51.441774 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f2d888-5d6f-4b80-8a00-4f87d0487541 service nova] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:38:51.442146 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f2d888-5d6f-4b80-8a00-4f87d0487541 service nova] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:38:51.442534 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f2d888-5d6f-4b80-8a00-4f87d0487541 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:38:51.444256 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-17f2d888-5d6f-4b80-8a00-4f87d0487541 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:38:51.538357 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f2d888-5d6f-4b80-8a00-4f87d0487541 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:38:51.540522 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f2d888-5d6f-4b80-8a00-4f87d0487541 service nova] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:38:51.559453 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 687/1360] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:38:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:38:53.769028 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b75f4d-fa95-4d65-b10d-e14569687d5e None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:38:53.769720 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b75f4d-fa95-4d65-b10d-e14569687d5e None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:38:53.769720 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b75f4d-fa95-4d65-b10d-e14569687d5e None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:38:53.771147 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 674/1361] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:38:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 13:38:53.776533 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291340fb-74bc-49d0-8e94-47c6bc0a8cb4 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:38:53.777028 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291340fb-74bc-49d0-8e94-47c6bc0a8cb4 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:38:53.777129 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291340fb-74bc-49d0-8e94-47c6bc0a8cb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:38:53.846110 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-291340fb-74bc-49d0-8e94-47c6bc0a8cb4 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:38:53.847316 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-291340fb-74bc-49d0-8e94-47c6bc0a8cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:38:53.886705 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:38:53.950479 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 688/1362] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:38:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:38:53.996890 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b75f4d-fa95-4d65-b10d-e14569687d5e None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:38:53.998810 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-c5b75f4d-fa95-4d65-b10d-e14569687d5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:38:54.093179 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:38:54.179290 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b75f4d-fa95-4d65-b10d-e14569687d5e None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:38:54.335524 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c761617-a9d3-4812-ac4e-62d2fab477d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c69161cd520b4b29bf50f3bf2f1c033b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:38:54.336830 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c761617-a9d3-4812-ac4e-62d2fab477d6 service nova] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:38:54.337204 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c761617-a9d3-4812-ac4e-62d2fab477d6 service nova] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:38:54.337581 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c761617-a9d3-4812-ac4e-62d2fab477d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:38:54.339340 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-9c761617-a9d3-4812-ac4e-62d2fab477d6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:38:54.365559 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:38:54.441183 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c761617-a9d3-4812-ac4e-62d2fab477d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:38:54.443201 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c761617-a9d3-4812-ac4e-62d2fab477d6 service nova] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:38:54.463685 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 675/1363] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:38:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:38:56.798820 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214c4a12-abcd-4d76-ac69-7d1328c281bb None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:38:56.799331 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214c4a12-abcd-4d76-ac69-7d1328c281bb None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:38:56.799331 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214c4a12-abcd-4d76-ac69-7d1328c281bb None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:38:56.800596 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 689/1364] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:38:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:38:56.806393 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c76a49-9c08-4374-a87c-3fead80337f0 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:38:56.806890 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c76a49-9c08-4374-a87c-3fead80337f0 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:38:56.806985 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c76a49-9c08-4374-a87c-3fead80337f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:38:56.884265 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-77c76a49-9c08-4374-a87c-3fead80337f0 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:38:56.885667 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-77c76a49-9c08-4374-a87c-3fead80337f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:38:56.995018 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:38:56.997822 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 676/1365] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:38:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 13:38:57.041733 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26edd8db-aeff-4991-b7ec-d03930a76dd4 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:38:57.042066 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26edd8db-aeff-4991-b7ec-d03930a76dd4 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:38:57.042300 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26edd8db-aeff-4991-b7ec-d03930a76dd4 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:38:57.045760 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 690/1366] 10.222.0.58 () {54 vars in 962 bytes} [Tue Jul 26 13:38:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:38:57.051466 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a81bead-5893-4f5f-b7f2-b616fb4961da None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:38:57.051682 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a81bead-5893-4f5f-b7f2-b616fb4961da None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:38:57.051949 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a81bead-5893-4f5f-b7f2-b616fb4961da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:38:57.131897 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-7a81bead-5893-4f5f-b7f2-b616fb4961da None None] MFA Rules not processed for user `c69161cd520b4b29bf50f3bf2f1c033b`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:38:57.133124 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-7a81bead-5893-4f5f-b7f2-b616fb4961da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:38:57.260996 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 677/1367] 10.222.0.58 () {58 vars in 1059 bytes} [Tue Jul 26 13:38:57 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:38:57.268823 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26edd8db-aeff-4991-b7ec-d03930a76dd4 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:38:57.270522 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-26edd8db-aeff-4991-b7ec-d03930a76dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:38:57.433351 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26edd8db-aeff-4991-b7ec-d03930a76dd4 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:38:57.454437 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:38:57.555294 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86513b44-065c-4ce6-a127-6728136666b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c69161cd520b4b29bf50f3bf2f1c033b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:38:57.555921 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86513b44-065c-4ce6-a127-6728136666b8 service nova] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:38:57.556207 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86513b44-065c-4ce6-a127-6728136666b8 service nova] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:38:57.556483 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86513b44-065c-4ce6-a127-6728136666b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:38:57.558156 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-86513b44-065c-4ce6-a127-6728136666b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:38:57.661167 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86513b44-065c-4ce6-a127-6728136666b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:38:57.662978 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86513b44-065c-4ce6-a127-6728136666b8 service nova] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:38:57.682228 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 691/1368] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:38:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:38:58.673345 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a784ecb2-16cd-44ee-b33f-1eb419596e9f None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:38:58.673951 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a784ecb2-16cd-44ee-b33f-1eb419596e9f None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:38:58.673951 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a784ecb2-16cd-44ee-b33f-1eb419596e9f None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:38:58.675485 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 678/1369] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:38:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:38:58.680949 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f56c918-029f-40a8-87f7-09f870efcc59 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:38:58.681346 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f56c918-029f-40a8-87f7-09f870efcc59 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:38:58.681559 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f56c918-029f-40a8-87f7-09f870efcc59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:38:58.713045 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:38:58.749357 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-9f56c918-029f-40a8-87f7-09f870efcc59 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:38:58.750627 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-9f56c918-029f-40a8-87f7-09f870efcc59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:38:58.850115 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 692/1370] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:38:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:38:59.192640 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd33c4e-c115-469e-ab6d-15a07acbb09e None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:38:59.192925 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd33c4e-c115-469e-ab6d-15a07acbb09e None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:38:59.193199 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd33c4e-c115-469e-ab6d-15a07acbb09e None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:38:59.195287 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 679/1371] 10.222.0.58 () {54 vars in 964 bytes} [Tue Jul 26 13:38:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:38:59.200010 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41c5edf-ba3f-4b3e-b7cc-8dc3e3247168 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:38:59.200303 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41c5edf-ba3f-4b3e-b7cc-8dc3e3247168 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:38:59.200611 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41c5edf-ba3f-4b3e-b7cc-8dc3e3247168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:38:59.289221 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-e41c5edf-ba3f-4b3e-b7cc-8dc3e3247168 None None] MFA Rules not processed for user `72e400f267fd488d8439426a81ba0a25`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:38:59.290786 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-e41c5edf-ba3f-4b3e-b7cc-8dc3e3247168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:38:59.343004 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:38:59.374177 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 693/1372] 10.222.0.58 () {58 vars in 1061 bytes} [Tue Jul 26 13:38:59 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:38:59.381585 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd33c4e-c115-469e-ab6d-15a07acbb09e None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:38:59.383631 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-2fd33c4e-c115-469e-ab6d-15a07acbb09e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:38:59.438464 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:38:59.495687 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd33c4e-c115-469e-ab6d-15a07acbb09e None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:38:59.574306 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e292163-0b0e-4457-a232-a6b513a1fd0d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:38:59.574853 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e292163-0b0e-4457-a232-a6b513a1fd0d service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:38:59.575137 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e292163-0b0e-4457-a232-a6b513a1fd0d service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:38:59.575410 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e292163-0b0e-4457-a232-a6b513a1fd0d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:38:59.577039 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-6e292163-0b0e-4457-a232-a6b513a1fd0d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:38:59.679066 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:38:59.697365 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e292163-0b0e-4457-a232-a6b513a1fd0d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:38:59.697365 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e292163-0b0e-4457-a232-a6b513a1fd0d service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:38:59.716071 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 680/1373] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:38:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:38:59.755545 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddae0dd-95d0-4206-84fe-532eeb7f2296 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:38:59.758379 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddae0dd-95d0-4206-84fe-532eeb7f2296 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:38:59.759267 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddae0dd-95d0-4206-84fe-532eeb7f2296 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:38:59.764664 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 694/1374] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:38:59 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:38:59.768426 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124b024a-532b-4e02-82a1-0f77d93e0875 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:38:59.768958 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124b024a-532b-4e02-82a1-0f77d93e0875 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:38:59.769530 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124b024a-532b-4e02-82a1-0f77d93e0875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:38:59.848664 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-124b024a-532b-4e02-82a1-0f77d93e0875 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:38:59.849905 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-124b024a-532b-4e02-82a1-0f77d93e0875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:38:59.948831 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 681/1375] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:38:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:38:59.992855 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddae0dd-95d0-4206-84fe-532eeb7f2296 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:38:59.995254 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-8ddae0dd-95d0-4206-84fe-532eeb7f2296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:39:00.176101 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddae0dd-95d0-4206-84fe-532eeb7f2296 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:39:00.228083 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:39:00.292011 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3315a7e-5d09-4cb7-af51-968bf78c7dad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c69161cd520b4b29bf50f3bf2f1c033b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:39:00.292640 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3315a7e-5d09-4cb7-af51-968bf78c7dad service nova] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:39:00.292922 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3315a7e-5d09-4cb7-af51-968bf78c7dad service nova] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:39:00.293191 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3315a7e-5d09-4cb7-af51-968bf78c7dad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:39:00.294871 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-b3315a7e-5d09-4cb7-af51-968bf78c7dad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:39:00.385940 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3315a7e-5d09-4cb7-af51-968bf78c7dad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:39:00.387548 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3315a7e-5d09-4cb7-af51-968bf78c7dad service nova] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:39:00.406385 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 695/1376] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:38:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:39:02.432780 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd6fc33-1a12-434f-93c7-dee857c3a368 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:39:02.433366 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd6fc33-1a12-434f-93c7-dee857c3a368 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:39:02.433366 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd6fc33-1a12-434f-93c7-dee857c3a368 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:39:02.434748 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 682/1377] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:39:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:39:02.440147 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26293d49-0101-4df4-94a3-cf19073bc55e None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:39:02.440604 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26293d49-0101-4df4-94a3-cf19073bc55e None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:39:02.440708 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26293d49-0101-4df4-94a3-cf19073bc55e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:39:02.507695 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-26293d49-0101-4df4-94a3-cf19073bc55e None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:39:02.508912 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-26293d49-0101-4df4-94a3-cf19073bc55e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:39:02.515635 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:39:02.616024 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 696/1378] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:39:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:39:02.662211 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd6fc33-1a12-434f-93c7-dee857c3a368 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:39:02.663942 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-8bd6fc33-1a12-434f-93c7-dee857c3a368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:39:02.735933 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:39:02.822843 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd6fc33-1a12-434f-93c7-dee857c3a368 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:39:02.934712 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e1b889-10ab-422d-920e-cee1bdf470fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c69161cd520b4b29bf50f3bf2f1c033b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:39:02.935335 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e1b889-10ab-422d-920e-cee1bdf470fb service nova] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:39:02.935619 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e1b889-10ab-422d-920e-cee1bdf470fb service nova] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:39:02.935896 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e1b889-10ab-422d-920e-cee1bdf470fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:39:02.937547 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-c0e1b889-10ab-422d-920e-cee1bdf470fb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:39:02.956377 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:39:03.051404 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e1b889-10ab-422d-920e-cee1bdf470fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:39:03.053064 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e1b889-10ab-422d-920e-cee1bdf470fb service nova] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:39:03.071370 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 683/1379] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:39:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:39:05.154785 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0de938-e8cb-4c0f-9283-16d797a92f28 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:39:05.155386 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0de938-e8cb-4c0f-9283-16d797a92f28 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:39:05.155386 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0de938-e8cb-4c0f-9283-16d797a92f28 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:39:05.157071 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 697/1380] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:39:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:39:05.164177 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89539f22-bc08-4f31-b9d4-7cf6531e25ba None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:39:05.164764 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89539f22-bc08-4f31-b9d4-7cf6531e25ba None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:39:05.164764 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89539f22-bc08-4f31-b9d4-7cf6531e25ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:39:05.238594 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-89539f22-bc08-4f31-b9d4-7cf6531e25ba None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:39:05.239833 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-89539f22-bc08-4f31-b9d4-7cf6531e25ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:39:05.337648 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:39:05.345515 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 684/1381] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:39:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:39:05.388449 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0de938-e8cb-4c0f-9283-16d797a92f28 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:39:05.390333 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-ef0de938-e8cb-4c0f-9283-16d797a92f28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:39:05.540921 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:39:05.552676 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0de938-e8cb-4c0f-9283-16d797a92f28 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:39:05.671660 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f77e8a-826c-47df-b83b-5e4b7c3193b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c69161cd520b4b29bf50f3bf2f1c033b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:39:05.672196 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f77e8a-826c-47df-b83b-5e4b7c3193b3 service nova] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:39:05.672478 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f77e8a-826c-47df-b83b-5e4b7c3193b3 service nova] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:39:05.672758 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f77e8a-826c-47df-b83b-5e4b7c3193b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:39:05.674486 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-67f77e8a-826c-47df-b83b-5e4b7c3193b3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:39:05.770266 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f77e8a-826c-47df-b83b-5e4b7c3193b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:39:05.771851 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f77e8a-826c-47df-b83b-5e4b7c3193b3 service nova] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:39:05.783676 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:39:05.792276 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 698/1382] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:39:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:39:07.879357 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dc6fc6-80be-4863-9f17-9cd019b55cb2 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:39:07.879920 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dc6fc6-80be-4863-9f17-9cd019b55cb2 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:39:07.880018 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dc6fc6-80be-4863-9f17-9cd019b55cb2 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:39:07.881618 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 685/1383] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:39:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 13:39:07.887115 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b80c79-aa92-4565-b892-41bbec25379c None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:39:07.887577 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b80c79-aa92-4565-b892-41bbec25379c None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:39:07.887674 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b80c79-aa92-4565-b892-41bbec25379c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:39:07.955331 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-d1b80c79-aa92-4565-b892-41bbec25379c None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:39:07.956562 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-d1b80c79-aa92-4565-b892-41bbec25379c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:39:08.058549 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 699/1384] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:39:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:39:08.100781 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dc6fc6-80be-4863-9f17-9cd019b55cb2 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:39:08.102377 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-30dc6fc6-80be-4863-9f17-9cd019b55cb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:39:08.270129 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dc6fc6-80be-4863-9f17-9cd019b55cb2 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:39:08.354402 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:39:08.408201 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f8d452-24d0-4cde-9637-1481456801df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c69161cd520b4b29bf50f3bf2f1c033b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:39:08.408784 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f8d452-24d0-4cde-9637-1481456801df service nova] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:39:08.409067 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f8d452-24d0-4cde-9637-1481456801df service nova] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:39:08.409344 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f8d452-24d0-4cde-9637-1481456801df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:39:08.411591 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-38f8d452-24d0-4cde-9637-1481456801df service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:39:08.525393 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f8d452-24d0-4cde-9637-1481456801df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:39:08.527673 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f8d452-24d0-4cde-9637-1481456801df service nova] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:39:08.549949 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 686/1385] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:39:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:39:11.020612 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ccd71f-dc85-4e6b-b810-cd963736579a None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:39:11.021150 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ccd71f-dc85-4e6b-b810-cd963736579a None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:39:11.021398 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ccd71f-dc85-4e6b-b810-cd963736579a None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:39:11.023727 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 700/1386] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:39:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:39:11.029883 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225a2bd2-ebdb-4149-8f92-d2f8afb94678 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:39:11.030326 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225a2bd2-ebdb-4149-8f92-d2f8afb94678 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:39:11.030567 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225a2bd2-ebdb-4149-8f92-d2f8afb94678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:39:11.096824 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-225a2bd2-ebdb-4149-8f92-d2f8afb94678 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:39:11.098091 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-225a2bd2-ebdb-4149-8f92-d2f8afb94678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:39:11.132209 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:39:11.204626 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 687/1387] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:39:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:39:11.250464 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ccd71f-dc85-4e6b-b810-cd963736579a None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:39:11.252216 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-24ccd71f-dc85-4e6b-b810-cd963736579a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:39:11.352922 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:39:11.444993 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ccd71f-dc85-4e6b-b810-cd963736579a None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:39:11.577638 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aa9698-9a9c-47e0-8a91-ca7e64c8ec80 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c69161cd520b4b29bf50f3bf2f1c033b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:39:11.578541 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aa9698-9a9c-47e0-8a91-ca7e64c8ec80 service nova] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:39:11.578881 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aa9698-9a9c-47e0-8a91-ca7e64c8ec80 service nova] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:39:11.579165 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aa9698-9a9c-47e0-8a91-ca7e64c8ec80 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:39:11.580842 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-b9aa9698-9a9c-47e0-8a91-ca7e64c8ec80 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:39:11.620154 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:39:11.672857 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9aa9698-9a9c-47e0-8a91-ca7e64c8ec80 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:39:11.674515 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9aa9698-9a9c-47e0-8a91-ca7e64c8ec80 service nova] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:39:11.693239 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 701/1388] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:39:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:39:13.837795 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cd53fd-8699-4bfc-9f1a-20a80c9f3fff None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:39:13.838364 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cd53fd-8699-4bfc-9f1a-20a80c9f3fff None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:39:13.838364 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cd53fd-8699-4bfc-9f1a-20a80c9f3fff None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:39:13.839760 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 688/1389] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:39:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:39:13.845258 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571e2258-cfb7-4a67-b6e9-984bcdcafd4e None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:39:13.845739 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571e2258-cfb7-4a67-b6e9-984bcdcafd4e None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:39:13.845838 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571e2258-cfb7-4a67-b6e9-984bcdcafd4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:39:13.912965 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-571e2258-cfb7-4a67-b6e9-984bcdcafd4e None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:39:13.914228 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-571e2258-cfb7-4a67-b6e9-984bcdcafd4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:39:14.008967 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 702/1390] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:39:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:39:14.052698 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cd53fd-8699-4bfc-9f1a-20a80c9f3fff None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:39:14.054777 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-35cd53fd-8699-4bfc-9f1a-20a80c9f3fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:39:14.202994 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cd53fd-8699-4bfc-9f1a-20a80c9f3fff None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:39:14.219424 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:39:14.318459 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db143bd-f6db-4f04-a0ef-42b2c0b7ecd3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c69161cd520b4b29bf50f3bf2f1c033b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:39:14.319072 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db143bd-f6db-4f04-a0ef-42b2c0b7ecd3 service nova] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:39:14.319863 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db143bd-f6db-4f04-a0ef-42b2c0b7ecd3 service nova] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:39:14.320135 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db143bd-f6db-4f04-a0ef-42b2c0b7ecd3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:39:14.321762 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-9db143bd-f6db-4f04-a0ef-42b2c0b7ecd3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:39:14.412246 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db143bd-f6db-4f04-a0ef-42b2c0b7ecd3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:39:14.414023 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db143bd-f6db-4f04-a0ef-42b2c0b7ecd3 service nova] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:39:14.432347 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 689/1391] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:39:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:39:15.411325 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f374b0-e7b9-477c-b1c3-06af69cceee9 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:39:15.411867 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f374b0-e7b9-477c-b1c3-06af69cceee9 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:39:15.411867 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f374b0-e7b9-477c-b1c3-06af69cceee9 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:39:15.413895 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 703/1392] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:39:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:39:15.419292 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56365fd5-7b8d-4184-904c-54cf1b82a3e5 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:39:15.419715 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56365fd5-7b8d-4184-904c-54cf1b82a3e5 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:39:15.419846 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56365fd5-7b8d-4184-904c-54cf1b82a3e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:39:15.446498 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:39:15.488357 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-56365fd5-7b8d-4184-904c-54cf1b82a3e5 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:39:15.489576 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-56365fd5-7b8d-4184-904c-54cf1b82a3e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:39:15.587471 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 690/1393] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:39:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:39:15.643298 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d667949d-c0f5-42fa-bccb-f1abf7dafc93 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:39:15.643577 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d667949d-c0f5-42fa-bccb-f1abf7dafc93 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:39:15.643861 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d667949d-c0f5-42fa-bccb-f1abf7dafc93 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:39:15.645218 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 704/1394] 10.222.0.58 () {54 vars in 966 bytes} [Tue Jul 26 13:39:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:39:15.650937 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156cf1c5-ab3b-4a83-bc9f-d2dda2ea9fc9 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:39:15.651364 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156cf1c5-ab3b-4a83-bc9f-d2dda2ea9fc9 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:39:15.651648 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156cf1c5-ab3b-4a83-bc9f-d2dda2ea9fc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:39:15.728045 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-156cf1c5-ab3b-4a83-bc9f-d2dda2ea9fc9 None None] MFA Rules not processed for user `eaf065aef9a046b2adaec34816a336b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:39:15.729148 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-156cf1c5-ab3b-4a83-bc9f-d2dda2ea9fc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:39:15.769584 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:39:15.805355 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 691/1395] 10.222.0.58 () {58 vars in 1063 bytes} [Tue Jul 26 13:39:15 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:39:15.812789 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d667949d-c0f5-42fa-bccb-f1abf7dafc93 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:39:15.814632 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-d667949d-c0f5-42fa-bccb-f1abf7dafc93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:39:15.828723 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:39:15.919791 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d667949d-c0f5-42fa-bccb-f1abf7dafc93 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:39:15.992458 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841d0085-0594-44f2-b36e-5b70e7e46e46 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf065aef9a046b2adaec34816a336b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:39:15.992982 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841d0085-0594-44f2-b36e-5b70e7e46e46 service glance] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:39:15.993259 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841d0085-0594-44f2-b36e-5b70e7e46e46 service glance] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:39:15.993555 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841d0085-0594-44f2-b36e-5b70e7e46e46 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:39:15.995157 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-841d0085-0594-44f2-b36e-5b70e7e46e46 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:39:16.068871 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:39:16.109986 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841d0085-0594-44f2-b36e-5b70e7e46e46 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:39:16.111414 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841d0085-0594-44f2-b36e-5b70e7e46e46 service glance] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:39:16.129740 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 705/1396] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:39:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:39:16.462503 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd84a9f-47ac-4940-b5bd-682356ea8ce6 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:39:16.463075 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd84a9f-47ac-4940-b5bd-682356ea8ce6 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:39:16.463075 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd84a9f-47ac-4940-b5bd-682356ea8ce6 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:39:16.464434 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 692/1397] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:39:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:39:16.469860 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1a226b-66d8-4f19-89a7-7e3b226759fe None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:39:16.470355 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1a226b-66d8-4f19-89a7-7e3b226759fe None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:39:16.470464 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1a226b-66d8-4f19-89a7-7e3b226759fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:39:16.541506 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-7d1a226b-66d8-4f19-89a7-7e3b226759fe None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:39:16.542656 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-7d1a226b-66d8-4f19-89a7-7e3b226759fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:39:16.643474 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 706/1398] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:39:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:39:16.687828 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd84a9f-47ac-4940-b5bd-682356ea8ce6 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:39:16.690303 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-7dd84a9f-47ac-4940-b5bd-682356ea8ce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:39:16.848525 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd84a9f-47ac-4940-b5bd-682356ea8ce6 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:39:16.861399 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66fc7df-30b1-49f7-9e50-dc74621b2525 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:39:16.861716 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66fc7df-30b1-49f7-9e50-dc74621b2525 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:39:16.861992 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66fc7df-30b1-49f7-9e50-dc74621b2525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:39:16.887869 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:39:16.899056 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:39:16.930021 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-b66fc7df-30b1-49f7-9e50-dc74621b2525 None None] MFA Rules not processed for user `b64bd7b183d24d8c9037b261c7884810`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:39:16.931244 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-b66fc7df-30b1-49f7-9e50-dc74621b2525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:39:16.967287 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fb4e59-db2c-46a5-a971-50c7d36966f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c69161cd520b4b29bf50f3bf2f1c033b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:39:16.967907 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fb4e59-db2c-46a5-a971-50c7d36966f4 service nova] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:39:16.968183 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fb4e59-db2c-46a5-a971-50c7d36966f4 service nova] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:39:16.968454 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fb4e59-db2c-46a5-a971-50c7d36966f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:39:16.970078 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-c0fb4e59-db2c-46a5-a971-50c7d36966f4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:39:17.017769 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 707/1399] 10.222.0.58 () {58 vars in 1009 bytes} [Tue Jul 26 13:39:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 13:39:17.064849 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fb4e59-db2c-46a5-a971-50c7d36966f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:39:17.066601 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fb4e59-db2c-46a5-a971-50c7d36966f4 service nova] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:39:17.067406 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eb256b-61a1-44a1-be1a-59e09b96500a None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:39:17.067715 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eb256b-61a1-44a1-be1a-59e09b96500a None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:39:17.067977 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eb256b-61a1-44a1-be1a-59e09b96500a None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:39:17.069377 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 708/1400] 10.222.0.58 () {54 vars in 943 bytes} [Tue Jul 26 13:39:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:39:17.075555 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c80fce1-3a3b-4222-a30f-33d1d07b829a None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:39:17.075969 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c80fce1-3a3b-4222-a30f-33d1d07b829a None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:39:17.076343 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c80fce1-3a3b-4222-a30f-33d1d07b829a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:39:17.086301 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 693/1401] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:39:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:39:17.158928 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-0c80fce1-3a3b-4222-a30f-33d1d07b829a None None] MFA Rules not processed for user `3ab2819bac844163ae4305e4f2186605`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:39:17.160172 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-0c80fce1-3a3b-4222-a30f-33d1d07b829a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:39:17.224447 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:39:17.285434 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 709/1402] 10.222.0.58 () {58 vars in 1040 bytes} [Tue Jul 26 13:39:17 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:39:17.295393 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fb4e59-db2c-46a5-a971-50c7d36966f4 service nova] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:39:17.296622 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-c0fb4e59-db2c-46a5-a971-50c7d36966f4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:39:17.352962 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:39:17.387477 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fb4e59-db2c-46a5-a971-50c7d36966f4 service nova] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:39:17.494665 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11460fc0-ab7c-48f6-83f0-3454938c1f48 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab2819bac844163ae4305e4f2186605', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:39:17.495314 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11460fc0-ab7c-48f6-83f0-3454938c1f48 service swift] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:39:17.495577 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11460fc0-ab7c-48f6-83f0-3454938c1f48 service swift] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:39:17.495812 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11460fc0-ab7c-48f6-83f0-3454938c1f48 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:39:17.497460 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-11460fc0-ab7c-48f6-83f0-3454938c1f48 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:39:17.582474 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11460fc0-ab7c-48f6-83f0-3454938c1f48 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:39:17.584087 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11460fc0-ab7c-48f6-83f0-3454938c1f48 service swift] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:39:17.588639 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:39:17.593106 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 694/1403] 10.222.0.58 () {60 vars in 1384 bytes} [Tue Jul 26 13:39:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 13:39:19.108779 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2bd147-cdaf-4e47-99d3-14d42b86479b None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:39:19.109389 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2bd147-cdaf-4e47-99d3-14d42b86479b None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:39:19.109389 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2bd147-cdaf-4e47-99d3-14d42b86479b None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:39:19.110855 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 710/1404] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:39:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:39:19.115900 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81658f17-90d4-4891-b7a6-810d8c3de837 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:39:19.116841 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81658f17-90d4-4891-b7a6-810d8c3de837 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:39:19.117004 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81658f17-90d4-4891-b7a6-810d8c3de837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:39:19.186975 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-81658f17-90d4-4891-b7a6-810d8c3de837 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:39:19.188180 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-81658f17-90d4-4891-b7a6-810d8c3de837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:39:19.296835 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 695/1405] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:39:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:39:19.341961 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2bd147-cdaf-4e47-99d3-14d42b86479b None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:39:19.343496 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-aa2bd147-cdaf-4e47-99d3-14d42b86479b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:39:19.505754 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2bd147-cdaf-4e47-99d3-14d42b86479b None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:39:19.534988 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:39:19.631576 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97f9634-b87d-455d-8121-7a0fc9457cd8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c69161cd520b4b29bf50f3bf2f1c033b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:39:19.632554 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97f9634-b87d-455d-8121-7a0fc9457cd8 service nova] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:39:19.632838 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97f9634-b87d-455d-8121-7a0fc9457cd8 service nova] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:39:19.633165 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97f9634-b87d-455d-8121-7a0fc9457cd8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:39:19.635150 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-a97f9634-b87d-455d-8121-7a0fc9457cd8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:39:19.733067 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97f9634-b87d-455d-8121-7a0fc9457cd8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:39:19.734795 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97f9634-b87d-455d-8121-7a0fc9457cd8 service nova] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:39:19.754941 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 711/1406] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:39:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:39:21.880610 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48f666c-6ec7-40ed-8d72-3b3140d990df None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:39:21.881224 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48f666c-6ec7-40ed-8d72-3b3140d990df None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:39:21.881224 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48f666c-6ec7-40ed-8d72-3b3140d990df None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:39:21.883933 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 696/1407] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:39:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:39:21.888284 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4786da-8745-4b28-8f3a-c44b8ff23f23 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:39:21.888724 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4786da-8745-4b28-8f3a-c44b8ff23f23 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:39:21.888813 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4786da-8745-4b28-8f3a-c44b8ff23f23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:39:21.937947 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:39:21.965636 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-6e4786da-8745-4b28-8f3a-c44b8ff23f23 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:39:21.966861 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-6e4786da-8745-4b28-8f3a-c44b8ff23f23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:39:22.073739 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 712/1408] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:39:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:39:22.117957 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48f666c-6ec7-40ed-8d72-3b3140d990df None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:39:22.119469 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-d48f666c-6ec7-40ed-8d72-3b3140d990df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:39:22.207126 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:39:22.279244 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48f666c-6ec7-40ed-8d72-3b3140d990df None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:39:22.407470 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212a0f84-cc78-47be-a32d-ce54264a4863 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c69161cd520b4b29bf50f3bf2f1c033b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:39:22.408098 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212a0f84-cc78-47be-a32d-ce54264a4863 service nova] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:39:22.408381 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212a0f84-cc78-47be-a32d-ce54264a4863 service nova] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:39:22.408656 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212a0f84-cc78-47be-a32d-ce54264a4863 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:39:22.410378 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-212a0f84-cc78-47be-a32d-ce54264a4863 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:39:22.457968 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:39:22.512508 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212a0f84-cc78-47be-a32d-ce54264a4863 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:39:22.514136 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212a0f84-cc78-47be-a32d-ce54264a4863 service nova] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:39:22.537523 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 697/1409] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:39:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:39:32.071771 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=51 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:39:32.534212 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=27 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:39:39.869307 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8907bd89-e314-4dbd-9b2e-9982a9838197 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:39:39.869900 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8907bd89-e314-4dbd-9b2e-9982a9838197 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:39:39.869900 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8907bd89-e314-4dbd-9b2e-9982a9838197 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:39:39.871284 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 713/1410] 10.222.0.58 () {54 vars in 918 bytes} [Tue Jul 26 13:39:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:39:39.876464 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e19bd9f-d8a7-4109-860d-f256ec368b38 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:39:39.876871 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e19bd9f-d8a7-4109-860d-f256ec368b38 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:39:39.877009 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e19bd9f-d8a7-4109-860d-f256ec368b38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:39:39.951772 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-8e19bd9f-d8a7-4109-860d-f256ec368b38 None None] MFA Rules not processed for user `835cbc13756b41b598142df3e35f5492`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:39:39.952958 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-8e19bd9f-d8a7-4109-860d-f256ec368b38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:39:40.057549 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 698/1411] 10.222.0.58 () {58 vars in 1015 bytes} [Tue Jul 26 13:39:39 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:39:40.072162 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93814a5-3de2-43d8-9b6d-f67d96855d96 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:39:40.072445 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93814a5-3de2-43d8-9b6d-f67d96855d96 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:39:40.072719 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93814a5-3de2-43d8-9b6d-f67d96855d96 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:39:40.074105 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 714/1412] 10.222.0.58 () {54 vars in 918 bytes} [Tue Jul 26 13:39:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:39:40.079029 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293d21e4-ff79-4342-acbf-4525f92c0108 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:39:40.079319 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293d21e4-ff79-4342-acbf-4525f92c0108 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:39:40.079603 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293d21e4-ff79-4342-acbf-4525f92c0108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:39:40.151776 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:39:40.154936 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-293d21e4-ff79-4342-acbf-4525f92c0108 None None] MFA Rules not processed for user `c69161cd520b4b29bf50f3bf2f1c033b`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:39:40.156664 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-293d21e4-ff79-4342-acbf-4525f92c0108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:39:40.271767 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 699/1413] 10.222.0.58 () {58 vars in 1015 bytes} [Tue Jul 26 13:39:40 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:39:42.728249 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae65a1d-33f8-45f0-b0fb-7c398d6af979 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:39:42.728762 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae65a1d-33f8-45f0-b0fb-7c398d6af979 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:39:42.728975 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae65a1d-33f8-45f0-b0fb-7c398d6af979 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:39:42.731467 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 715/1414] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 13:39:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:39:42.736741 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31d471a-d8d6-4e71-ae98-0c2ece67c5bd None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:39:42.737144 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31d471a-d8d6-4e71-ae98-0c2ece67c5bd None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:39:42.737330 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31d471a-d8d6-4e71-ae98-0c2ece67c5bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:39:42.802985 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-d31d471a-d8d6-4e71-ae98-0c2ece67c5bd None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:39:42.804185 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-d31d471a-d8d6-4e71-ae98-0c2ece67c5bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:39:42.861040 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:39:42.903637 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 700/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 13:39:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:39:42.947154 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae65a1d-33f8-45f0-b0fb-7c398d6af979 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:39:42.948676 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-7ae65a1d-33f8-45f0-b0fb-7c398d6af979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:39:43.092336 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae65a1d-33f8-45f0-b0fb-7c398d6af979 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:39:43.163847 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:39:43.204520 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a187f5-7b17-459b-aca2-f4983b5710d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c69161cd520b4b29bf50f3bf2f1c033b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:39:43.205248 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a187f5-7b17-459b-aca2-f4983b5710d6 service nova] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:39:43.205561 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a187f5-7b17-459b-aca2-f4983b5710d6 service nova] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:39:43.205839 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a187f5-7b17-459b-aca2-f4983b5710d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:39:43.207484 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-93a187f5-7b17-459b-aca2-f4983b5710d6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:39:43.293975 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a187f5-7b17-459b-aca2-f4983b5710d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:39:43.295498 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a187f5-7b17-459b-aca2-f4983b5710d6 service nova] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:39:43.313454 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 716/1416] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:39:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:39:44.204904 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2844f8a5-0c40-479f-aedf-663f319b0815 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:39:44.205496 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2844f8a5-0c40-479f-aedf-663f319b0815 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:39:44.205496 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2844f8a5-0c40-479f-aedf-663f319b0815 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:39:44.206878 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 701/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 13:39:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:39:44.212523 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8237d42b-e9f4-4235-8166-18a1c26c60d4 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:39:44.212979 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8237d42b-e9f4-4235-8166-18a1c26c60d4 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:39:44.213096 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8237d42b-e9f4-4235-8166-18a1c26c60d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:39:44.287326 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-8237d42b-e9f4-4235-8166-18a1c26c60d4 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:39:44.288520 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-8237d42b-e9f4-4235-8166-18a1c26c60d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:39:44.314514 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:39:44.382352 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 717/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 13:39:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:39:44.535472 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2844f8a5-0c40-479f-aedf-663f319b0815 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:39:44.537258 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-2844f8a5-0c40-479f-aedf-663f319b0815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:39:44.680749 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2844f8a5-0c40-479f-aedf-663f319b0815 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:39:44.721261 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:39:44.792997 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997e6c70-8a1a-473e-a713-edb43fded5ce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c69161cd520b4b29bf50f3bf2f1c033b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:39:44.793657 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997e6c70-8a1a-473e-a713-edb43fded5ce service nova] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:39:44.793976 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997e6c70-8a1a-473e-a713-edb43fded5ce service nova] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:39:44.794402 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997e6c70-8a1a-473e-a713-edb43fded5ce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:39:44.796055 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-997e6c70-8a1a-473e-a713-edb43fded5ce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:39:44.887879 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997e6c70-8a1a-473e-a713-edb43fded5ce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:39:44.889461 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997e6c70-8a1a-473e-a713-edb43fded5ce service nova] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:39:44.907141 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 702/1419] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:39:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:39:45.784573 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd71f866-c473-42eb-b8fd-62f01f4d87de None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:39:45.785143 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd71f866-c473-42eb-b8fd-62f01f4d87de None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:39:45.785143 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd71f866-c473-42eb-b8fd-62f01f4d87de None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:39:45.786807 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 718/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 13:39:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:39:45.791775 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08fcc21-f33a-4f33-b012-ce490b11d2e6 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:39:45.792225 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08fcc21-f33a-4f33-b012-ce490b11d2e6 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:39:45.792321 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08fcc21-f33a-4f33-b012-ce490b11d2e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:39:45.845621 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:39:45.867999 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-f08fcc21-f33a-4f33-b012-ce490b11d2e6 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:39:45.869207 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-f08fcc21-f33a-4f33-b012-ce490b11d2e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:39:45.969603 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 703/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 13:39:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:39:46.123897 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd71f866-c473-42eb-b8fd-62f01f4d87de None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:39:46.125457 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-bd71f866-c473-42eb-b8fd-62f01f4d87de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:39:46.269446 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd71f866-c473-42eb-b8fd-62f01f4d87de None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:39:46.281364 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:39:46.382257 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fe7559-9535-495b-88dc-3aa15c0133d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c69161cd520b4b29bf50f3bf2f1c033b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:39:46.382885 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fe7559-9535-495b-88dc-3aa15c0133d9 service nova] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:39:46.383174 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fe7559-9535-495b-88dc-3aa15c0133d9 service nova] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:39:46.383475 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fe7559-9535-495b-88dc-3aa15c0133d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:39:46.385109 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-a5fe7559-9535-495b-88dc-3aa15c0133d9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:39:46.473371 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fe7559-9535-495b-88dc-3aa15c0133d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:39:46.474983 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fe7559-9535-495b-88dc-3aa15c0133d9 service nova] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:39:46.492570 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 719/1422] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:39:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:39:47.353849 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1ed1ea-a85a-4c33-9039-1f6e973f21b7 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:39:47.354393 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1ed1ea-a85a-4c33-9039-1f6e973f21b7 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:39:47.354393 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1ed1ea-a85a-4c33-9039-1f6e973f21b7 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:39:47.355772 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 704/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 13:39:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:39:47.360619 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9666e040-7a79-467f-b06e-6c0135d9fdf7 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:39:47.361015 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9666e040-7a79-467f-b06e-6c0135d9fdf7 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:39:47.361170 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9666e040-7a79-467f-b06e-6c0135d9fdf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:39:47.381042 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:39:47.439229 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-9666e040-7a79-467f-b06e-6c0135d9fdf7 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:39:47.440433 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-9666e040-7a79-467f-b06e-6c0135d9fdf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:39:47.537125 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 720/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 13:39:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:39:47.686501 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1ed1ea-a85a-4c33-9039-1f6e973f21b7 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:39:47.687951 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-af1ed1ea-a85a-4c33-9039-1f6e973f21b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:39:47.796734 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:39:47.831765 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1ed1ea-a85a-4c33-9039-1f6e973f21b7 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:39:48.063777 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18abb19e-ed1e-4126-8c11-62baaa01d482 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c69161cd520b4b29bf50f3bf2f1c033b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:39:48.064386 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18abb19e-ed1e-4126-8c11-62baaa01d482 service nova] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:39:48.064662 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18abb19e-ed1e-4126-8c11-62baaa01d482 service nova] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:39:48.064930 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18abb19e-ed1e-4126-8c11-62baaa01d482 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:39:48.066553 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-18abb19e-ed1e-4126-8c11-62baaa01d482 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:39:48.141469 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:39:48.156088 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18abb19e-ed1e-4126-8c11-62baaa01d482 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:39:48.157696 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18abb19e-ed1e-4126-8c11-62baaa01d482 service nova] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:39:48.175616 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 705/1425] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:39:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:39:49.039382 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b081e3-abe5-4e4f-9b25-690f7427c20d None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:39:49.039862 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b081e3-abe5-4e4f-9b25-690f7427c20d None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:39:49.040448 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b081e3-abe5-4e4f-9b25-690f7427c20d None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:39:49.041445 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 721/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 13:39:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:39:49.046505 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23672ded-296d-4f83-a30b-585ddfee0942 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:39:49.046930 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23672ded-296d-4f83-a30b-585ddfee0942 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:39:49.047045 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23672ded-296d-4f83-a30b-585ddfee0942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:39:49.122808 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-23672ded-296d-4f83-a30b-585ddfee0942 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:39:49.123998 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-23672ded-296d-4f83-a30b-585ddfee0942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:39:49.218803 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 706/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 13:39:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 13:39:49.369937 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b081e3-abe5-4e4f-9b25-690f7427c20d None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:39:49.371453 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-d4b081e3-abe5-4e4f-9b25-690f7427c20d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:39:49.452702 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:39:49.513298 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b081e3-abe5-4e4f-9b25-690f7427c20d None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:39:49.622469 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec2e564-8a58-4f90-a7fd-20100c21c89d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c69161cd520b4b29bf50f3bf2f1c033b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:39:49.623019 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec2e564-8a58-4f90-a7fd-20100c21c89d service nova] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:39:49.623302 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec2e564-8a58-4f90-a7fd-20100c21c89d service nova] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:39:49.623579 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec2e564-8a58-4f90-a7fd-20100c21c89d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:39:49.625210 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-cec2e564-8a58-4f90-a7fd-20100c21c89d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:39:49.673482 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:39:49.714767 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec2e564-8a58-4f90-a7fd-20100c21c89d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:39:49.716355 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec2e564-8a58-4f90-a7fd-20100c21c89d service nova] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:39:49.733720 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 722/1428] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:39:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:39:50.532089 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e67070-cb72-491e-b681-2cc0e2569403 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:39:50.532633 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e67070-cb72-491e-b681-2cc0e2569403 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:39:50.532633 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e67070-cb72-491e-b681-2cc0e2569403 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:39:50.534047 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 707/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 13:39:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:39:50.539209 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c127d076-14ec-43c4-9ff3-2fac4a55121b None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:39:50.539677 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c127d076-14ec-43c4-9ff3-2fac4a55121b None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:39:50.539778 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c127d076-14ec-43c4-9ff3-2fac4a55121b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:39:50.614608 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-c127d076-14ec-43c4-9ff3-2fac4a55121b None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:39:50.615808 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-c127d076-14ec-43c4-9ff3-2fac4a55121b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:39:50.709912 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 723/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 13:39:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:39:50.866032 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e67070-cb72-491e-b681-2cc0e2569403 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:39:50.867548 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-06e67070-cb72-491e-b681-2cc0e2569403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:39:50.917595 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:39:51.012750 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e67070-cb72-491e-b681-2cc0e2569403 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:39:51.125096 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f357073-42a9-4d94-bb36-c26685a7f5d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c69161cd520b4b29bf50f3bf2f1c033b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:39:51.125681 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f357073-42a9-4d94-bb36-c26685a7f5d2 service nova] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:39:51.125963 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f357073-42a9-4d94-bb36-c26685a7f5d2 service nova] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:39:51.126237 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f357073-42a9-4d94-bb36-c26685a7f5d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:39:51.127846 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-7f357073-42a9-4d94-bb36-c26685a7f5d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:39:51.134756 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:39:51.216082 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f357073-42a9-4d94-bb36-c26685a7f5d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:39:51.217682 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f357073-42a9-4d94-bb36-c26685a7f5d2 service nova] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:39:51.235720 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 708/1431] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:39:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:39:52.027665 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13275dd-8a96-4347-bf73-c8156ea20675 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:39:52.028246 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13275dd-8a96-4347-bf73-c8156ea20675 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:39:52.028246 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13275dd-8a96-4347-bf73-c8156ea20675 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:39:52.029577 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 724/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 13:39:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:39:52.034373 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aace75-2bbe-4249-89f1-b13da63857d6 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:39:52.034831 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aace75-2bbe-4249-89f1-b13da63857d6 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:39:52.034937 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aace75-2bbe-4249-89f1-b13da63857d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:39:52.109891 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-a4aace75-2bbe-4249-89f1-b13da63857d6 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:39:52.111092 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-a4aace75-2bbe-4249-89f1-b13da63857d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:39:52.179750 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:39:52.207959 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 709/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 13:39:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:39:52.358400 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13275dd-8a96-4347-bf73-c8156ea20675 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:39:52.359954 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-e13275dd-8a96-4347-bf73-c8156ea20675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:39:52.376935 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:39:52.501314 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13275dd-8a96-4347-bf73-c8156ea20675 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:39:52.588263 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:39:52.614033 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced808e0-54cb-43af-b2f5-53458d2bbf51 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c69161cd520b4b29bf50f3bf2f1c033b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:39:52.614589 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced808e0-54cb-43af-b2f5-53458d2bbf51 service nova] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:39:52.614867 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced808e0-54cb-43af-b2f5-53458d2bbf51 service nova] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:39:52.615137 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced808e0-54cb-43af-b2f5-53458d2bbf51 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:39:52.616735 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-ced808e0-54cb-43af-b2f5-53458d2bbf51 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:39:52.703704 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced808e0-54cb-43af-b2f5-53458d2bbf51 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:39:52.705274 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced808e0-54cb-43af-b2f5-53458d2bbf51 service nova] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:39:52.723176 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 725/1434] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:39:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:39:53.516946 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e99cccf-53fc-4301-aa94-11f4d2b43f8b None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:39:53.517588 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e99cccf-53fc-4301-aa94-11f4d2b43f8b None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:39:53.517588 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e99cccf-53fc-4301-aa94-11f4d2b43f8b None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:39:53.518953 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 710/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 13:39:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:39:53.523713 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb07aef8-e842-4e06-82e4-a41857ec584f None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:39:53.524192 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb07aef8-e842-4e06-82e4-a41857ec584f None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:39:53.524313 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb07aef8-e842-4e06-82e4-a41857ec584f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:39:53.598881 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-eb07aef8-e842-4e06-82e4-a41857ec584f None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:39:53.600067 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-eb07aef8-e842-4e06-82e4-a41857ec584f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:39:53.640563 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:39:53.694014 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 726/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 13:39:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:39:53.842147 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e99cccf-53fc-4301-aa94-11f4d2b43f8b None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:39:53.843693 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-1e99cccf-53fc-4301-aa94-11f4d2b43f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:39:53.987749 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e99cccf-53fc-4301-aa94-11f4d2b43f8b None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:39:54.041700 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:39:54.099495 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1498147-2f83-43ea-b685-b643fde1aef2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c69161cd520b4b29bf50f3bf2f1c033b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:39:54.100144 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1498147-2f83-43ea-b685-b643fde1aef2 service nova] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:39:54.100425 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1498147-2f83-43ea-b685-b643fde1aef2 service nova] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:39:54.100700 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1498147-2f83-43ea-b685-b643fde1aef2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:39:54.102864 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-f1498147-2f83-43ea-b685-b643fde1aef2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:39:54.190701 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1498147-2f83-43ea-b685-b643fde1aef2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:39:54.192275 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1498147-2f83-43ea-b685-b643fde1aef2 service nova] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:39:54.209780 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 711/1437] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:39:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:39:55.010711 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e73fef-3718-4e28-8855-6ead0faeb8d6 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:39:55.011312 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e73fef-3718-4e28-8855-6ead0faeb8d6 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:39:55.011312 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e73fef-3718-4e28-8855-6ead0faeb8d6 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:39:55.012609 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 727/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 13:39:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:39:55.018094 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c6bb10-240c-40e1-b82f-d8db7f852f4c None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:39:55.018503 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c6bb10-240c-40e1-b82f-d8db7f852f4c None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:39:55.018677 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c6bb10-240c-40e1-b82f-d8db7f852f4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:39:55.094279 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-37c6bb10-240c-40e1-b82f-d8db7f852f4c None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:39:55.095473 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-37c6bb10-240c-40e1-b82f-d8db7f852f4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:39:55.097457 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:39:55.191157 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 712/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 13:39:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:39:55.341147 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e73fef-3718-4e28-8855-6ead0faeb8d6 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:39:55.342698 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-06e73fef-3718-4e28-8855-6ead0faeb8d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:39:55.483871 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e73fef-3718-4e28-8855-6ead0faeb8d6 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:39:55.509826 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:39:55.593075 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa82757-aa79-48c4-93aa-a4de33260bd2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c69161cd520b4b29bf50f3bf2f1c033b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:39:55.593715 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa82757-aa79-48c4-93aa-a4de33260bd2 service nova] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:39:55.593996 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa82757-aa79-48c4-93aa-a4de33260bd2 service nova] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:39:55.594268 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa82757-aa79-48c4-93aa-a4de33260bd2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:39:55.595878 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-cfa82757-aa79-48c4-93aa-a4de33260bd2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:39:55.682715 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa82757-aa79-48c4-93aa-a4de33260bd2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:39:55.684292 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa82757-aa79-48c4-93aa-a4de33260bd2 service nova] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:39:55.701664 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 728/1440] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:39:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:39:57.597968 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c46430-f0c8-42d1-9ba8-3d0241dd6100 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:39:57.598445 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c46430-f0c8-42d1-9ba8-3d0241dd6100 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:39:57.598549 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c46430-f0c8-42d1-9ba8-3d0241dd6100 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:39:57.599863 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 713/1441] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 13:39:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:39:57.605278 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be25d8a2-f798-435c-8980-f47a62a0ddec None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:39:57.605745 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be25d8a2-f798-435c-8980-f47a62a0ddec None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:39:57.605854 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be25d8a2-f798-435c-8980-f47a62a0ddec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:39:57.646061 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:39:57.671766 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-be25d8a2-f798-435c-8980-f47a62a0ddec None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:39:57.672954 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-be25d8a2-f798-435c-8980-f47a62a0ddec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:39:57.770399 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 729/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 13:39:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:39:57.809719 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c46430-f0c8-42d1-9ba8-3d0241dd6100 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:39:57.811237 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-84c46430-f0c8-42d1-9ba8-3d0241dd6100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:39:57.914473 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c46430-f0c8-42d1-9ba8-3d0241dd6100 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:39:57.946704 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:39:57.988035 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b57eac3-4b1d-4e3a-abcf-55b1b412880c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:39:57.988633 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b57eac3-4b1d-4e3a-abcf-55b1b412880c service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:39:57.988910 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b57eac3-4b1d-4e3a-abcf-55b1b412880c service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:39:57.989177 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b57eac3-4b1d-4e3a-abcf-55b1b412880c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:39:57.990790 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-2b57eac3-4b1d-4e3a-abcf-55b1b412880c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:39:58.099733 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b57eac3-4b1d-4e3a-abcf-55b1b412880c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:39:58.101202 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b57eac3-4b1d-4e3a-abcf-55b1b412880c service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:39:58.118880 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 714/1443] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:39:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:40:00.250735 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bba2ac-7099-46bc-a9e1-55075de609ec None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:40:00.251306 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bba2ac-7099-46bc-a9e1-55075de609ec None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:40:00.251306 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bba2ac-7099-46bc-a9e1-55075de609ec None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:40:00.252631 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 730/1444] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 13:40:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:40:00.257976 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c3b9d9-90ac-42d4-8843-6915d70dc5de None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:40:00.258413 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c3b9d9-90ac-42d4-8843-6915d70dc5de None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:40:00.258610 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c3b9d9-90ac-42d4-8843-6915d70dc5de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:40:00.324154 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-22c3b9d9-90ac-42d4-8843-6915d70dc5de None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:40:00.325272 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-22c3b9d9-90ac-42d4-8843-6915d70dc5de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:40:00.338256 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:40:00.423475 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 715/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 13:40:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:40:00.460504 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bba2ac-7099-46bc-a9e1-55075de609ec None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:40:00.462586 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-e4bba2ac-7099-46bc-a9e1-55075de609ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:40:00.564663 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bba2ac-7099-46bc-a9e1-55075de609ec None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:40:00.577306 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:40:00.637438 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30291a11-e948-43ba-8f4f-0ac8c1ad7668 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:40:00.638083 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30291a11-e948-43ba-8f4f-0ac8c1ad7668 service neutron] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:40:00.638366 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30291a11-e948-43ba-8f4f-0ac8c1ad7668 service neutron] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:40:00.638642 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30291a11-e948-43ba-8f4f-0ac8c1ad7668 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:40:00.640262 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-30291a11-e948-43ba-8f4f-0ac8c1ad7668 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:40:00.746599 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30291a11-e948-43ba-8f4f-0ac8c1ad7668 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:40:00.748014 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30291a11-e948-43ba-8f4f-0ac8c1ad7668 service neutron] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:40:00.766975 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 731/1446] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:40:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:40:03.040991 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26de34ae-e6fd-4941-bc49-fbc6b1a48fda None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:40:03.041387 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26de34ae-e6fd-4941-bc49-fbc6b1a48fda None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:40:03.041560 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26de34ae-e6fd-4941-bc49-fbc6b1a48fda None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:40:03.042921 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 716/1447] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 13:40:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:40:03.048128 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac91fad-406f-43bb-a595-76444dfdf898 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:40:03.048565 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac91fad-406f-43bb-a595-76444dfdf898 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:40:03.048710 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac91fad-406f-43bb-a595-76444dfdf898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:40:03.095875 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:40:03.114250 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-7ac91fad-406f-43bb-a595-76444dfdf898 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:40:03.115459 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-7ac91fad-406f-43bb-a595-76444dfdf898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:40:03.212617 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 732/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 13:40:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 13:40:03.250984 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26de34ae-e6fd-4941-bc49-fbc6b1a48fda None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:40:03.252502 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-26de34ae-e6fd-4941-bc49-fbc6b1a48fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:40:03.354865 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26de34ae-e6fd-4941-bc49-fbc6b1a48fda None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:40:03.394598 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:40:03.428000 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1c8bd7-e590-40e7-bba3-1e804cfe03c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:40:03.428608 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1c8bd7-e590-40e7-bba3-1e804cfe03c9 service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:40:03.428885 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1c8bd7-e590-40e7-bba3-1e804cfe03c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:40:03.429156 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1c8bd7-e590-40e7-bba3-1e804cfe03c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:40:03.431269 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-cf1c8bd7-e590-40e7-bba3-1e804cfe03c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:40:03.538733 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1c8bd7-e590-40e7-bba3-1e804cfe03c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:40:03.540736 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1c8bd7-e590-40e7-bba3-1e804cfe03c9 service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:40:03.558270 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 717/1449] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:40:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:40:05.683783 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45e7177-1724-4e46-9dab-96d1b71b9690 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:40:05.684343 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45e7177-1724-4e46-9dab-96d1b71b9690 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:40:05.684343 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45e7177-1724-4e46-9dab-96d1b71b9690 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:40:05.685740 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 733/1450] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 13:40:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:40:05.691138 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce646acd-fc6c-4d54-a595-10ae3f59aa28 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:40:05.691554 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce646acd-fc6c-4d54-a595-10ae3f59aa28 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:40:05.691688 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce646acd-fc6c-4d54-a595-10ae3f59aa28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:40:05.756419 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-ce646acd-fc6c-4d54-a595-10ae3f59aa28 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:40:05.757640 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-ce646acd-fc6c-4d54-a595-10ae3f59aa28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:40:05.777859 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:40:05.853365 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 718/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 13:40:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:40:05.892317 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45e7177-1724-4e46-9dab-96d1b71b9690 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:40:05.893885 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-c45e7177-1724-4e46-9dab-96d1b71b9690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:40:05.993854 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45e7177-1724-4e46-9dab-96d1b71b9690 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:40:06.013045 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:40:06.066141 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10504f91-5307-49f9-a614-21b7b8ed8ab1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:40:06.066755 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10504f91-5307-49f9-a614-21b7b8ed8ab1 service neutron] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:40:06.067032 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10504f91-5307-49f9-a614-21b7b8ed8ab1 service neutron] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:40:06.067303 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10504f91-5307-49f9-a614-21b7b8ed8ab1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:40:06.068905 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-10504f91-5307-49f9-a614-21b7b8ed8ab1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:40:06.177109 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10504f91-5307-49f9-a614-21b7b8ed8ab1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:40:06.178567 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10504f91-5307-49f9-a614-21b7b8ed8ab1 service neutron] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:40:06.196785 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 734/1452] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:40:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:40:08.252878 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7562f684-275f-420f-a04c-462627114604 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:40:08.253447 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7562f684-275f-420f-a04c-462627114604 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:40:08.253447 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7562f684-275f-420f-a04c-462627114604 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:40:08.254817 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 719/1453] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 13:40:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:40:08.260580 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0038da52-0734-44f2-8c70-b31dba859dd5 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:40:08.261017 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0038da52-0734-44f2-8c70-b31dba859dd5 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:40:08.261136 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0038da52-0734-44f2-8c70-b31dba859dd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:40:08.323966 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:40:08.326720 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-0038da52-0734-44f2-8c70-b31dba859dd5 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:40:08.328474 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-0038da52-0734-44f2-8c70-b31dba859dd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:40:08.423401 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 735/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 13:40:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:40:08.462136 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7562f684-275f-420f-a04c-462627114604 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:40:08.463750 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-7562f684-275f-420f-a04c-462627114604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:40:08.565342 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7562f684-275f-420f-a04c-462627114604 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:40:08.613197 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:40:08.638959 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abc257e-29bd-4a8a-ab15-f128fb8584b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:40:08.639565 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc257e-29bd-4a8a-ab15-f128fb8584b0 service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:40:08.639846 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc257e-29bd-4a8a-ab15-f128fb8584b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:40:08.640122 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc257e-29bd-4a8a-ab15-f128fb8584b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:40:08.641760 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-9abc257e-29bd-4a8a-ab15-f128fb8584b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:40:08.749332 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abc257e-29bd-4a8a-ab15-f128fb8584b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:40:08.750787 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abc257e-29bd-4a8a-ab15-f128fb8584b0 service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:40:08.769334 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 720/1455] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:40:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:40:10.816540 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17054e09-2a0f-4290-b8c9-f015499b68a2 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:40:10.817115 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17054e09-2a0f-4290-b8c9-f015499b68a2 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:40:10.817115 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17054e09-2a0f-4290-b8c9-f015499b68a2 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:40:10.818473 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 736/1456] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 13:40:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:40:10.823729 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7c13f8-4cca-4359-bf11-d807cb8fdd2f None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:40:10.824197 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7c13f8-4cca-4359-bf11-d807cb8fdd2f None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:40:10.824303 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7c13f8-4cca-4359-bf11-d807cb8fdd2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:40:10.889685 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-5d7c13f8-4cca-4359-bf11-d807cb8fdd2f None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:40:10.890883 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-5d7c13f8-4cca-4359-bf11-d807cb8fdd2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:40:10.920253 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:40:10.990739 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 721/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 13:40:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:40:11.028447 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17054e09-2a0f-4290-b8c9-f015499b68a2 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:40:11.030023 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-17054e09-2a0f-4290-b8c9-f015499b68a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:40:11.133605 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17054e09-2a0f-4290-b8c9-f015499b68a2 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:40:11.281880 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:40:11.327451 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08067cfc-5d92-4bcc-9331-7772136324bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:40:11.328066 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08067cfc-5d92-4bcc-9331-7772136324bc service neutron] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:40:11.328345 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08067cfc-5d92-4bcc-9331-7772136324bc service neutron] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:40:11.328620 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08067cfc-5d92-4bcc-9331-7772136324bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:40:11.330274 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-08067cfc-5d92-4bcc-9331-7772136324bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:40:11.437104 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08067cfc-5d92-4bcc-9331-7772136324bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:40:11.438504 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08067cfc-5d92-4bcc-9331-7772136324bc service neutron] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:40:11.456470 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 737/1458] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:40:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:40:13.518555 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee5445b-9b88-4d4e-82e6-6363d52a168c None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:40:13.519097 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee5445b-9b88-4d4e-82e6-6363d52a168c None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:40:13.519097 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee5445b-9b88-4d4e-82e6-6363d52a168c None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:40:13.520595 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 722/1459] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 13:40:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:40:13.526330 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f27aad3-c6f8-447f-8d2e-6c24eb306f5c None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:40:13.526707 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f27aad3-c6f8-447f-8d2e-6c24eb306f5c None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:40:13.526888 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f27aad3-c6f8-447f-8d2e-6c24eb306f5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:40:13.590774 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-5f27aad3-c6f8-447f-8d2e-6c24eb306f5c None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:40:13.591963 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-5f27aad3-c6f8-447f-8d2e-6c24eb306f5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:40:13.600074 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:40:13.685908 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 738/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 13:40:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:40:13.723538 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee5445b-9b88-4d4e-82e6-6363d52a168c None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:40:13.725054 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-cee5445b-9b88-4d4e-82e6-6363d52a168c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:40:13.827736 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee5445b-9b88-4d4e-82e6-6363d52a168c None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:40:13.880840 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:40:13.899705 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93f87c4-0d8d-46bd-a5ac-134b28429e1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:40:13.900319 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93f87c4-0d8d-46bd-a5ac-134b28429e1f service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:40:13.900614 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93f87c4-0d8d-46bd-a5ac-134b28429e1f service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:40:13.900874 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93f87c4-0d8d-46bd-a5ac-134b28429e1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:40:13.902518 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-b93f87c4-0d8d-46bd-a5ac-134b28429e1f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:40:14.009891 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93f87c4-0d8d-46bd-a5ac-134b28429e1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:40:14.011340 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93f87c4-0d8d-46bd-a5ac-134b28429e1f service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:40:14.029751 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 723/1461] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:40:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:40:16.041361 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2644f56e-0ce7-414c-9b9c-febe0b8913dd None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:40:16.041361 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2644f56e-0ce7-414c-9b9c-febe0b8913dd None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:40:16.041361 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2644f56e-0ce7-414c-9b9c-febe0b8913dd None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:40:16.042802 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 739/1462] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 13:40:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:40:16.047977 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ead0fa-6c74-4049-a150-196c64f1bd4e None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:40:16.048378 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ead0fa-6c74-4049-a150-196c64f1bd4e None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:40:16.048536 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ead0fa-6c74-4049-a150-196c64f1bd4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:40:16.114945 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-89ead0fa-6c74-4049-a150-196c64f1bd4e None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:40:16.116153 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-89ead0fa-6c74-4049-a150-196c64f1bd4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:40:16.154162 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:40:16.212621 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 724/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 13:40:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:40:16.250226 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2644f56e-0ce7-414c-9b9c-febe0b8913dd None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:40:16.251810 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-2644f56e-0ce7-414c-9b9c-febe0b8913dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:40:16.354341 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2644f56e-0ce7-414c-9b9c-febe0b8913dd None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:40:16.391596 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:40:16.428729 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c399ca2-0b9a-4acc-b3b6-42a1c0846f9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:40:16.429335 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c399ca2-0b9a-4acc-b3b6-42a1c0846f9e service neutron] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:40:16.429652 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c399ca2-0b9a-4acc-b3b6-42a1c0846f9e service neutron] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:40:16.429948 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c399ca2-0b9a-4acc-b3b6-42a1c0846f9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:40:16.432070 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-7c399ca2-0b9a-4acc-b3b6-42a1c0846f9e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:40:16.534960 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c399ca2-0b9a-4acc-b3b6-42a1c0846f9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:40:16.536622 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c399ca2-0b9a-4acc-b3b6-42a1c0846f9e service neutron] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:40:16.555861 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 740/1464] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:40:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:40:18.547955 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db5d5e0-2946-41cb-8a4f-03723c98e622 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:40:18.548525 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db5d5e0-2946-41cb-8a4f-03723c98e622 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:40:18.548525 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db5d5e0-2946-41cb-8a4f-03723c98e622 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:40:18.549874 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 725/1465] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 13:40:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:40:18.555487 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca39d56-9533-4002-969f-46c547e878f1 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:40:18.555923 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca39d56-9533-4002-969f-46c547e878f1 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:40:18.556023 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca39d56-9533-4002-969f-46c547e878f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:40:18.619868 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-5ca39d56-9533-4002-969f-46c547e878f1 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:40:18.621072 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-5ca39d56-9533-4002-969f-46c547e878f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:40:18.638890 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:40:18.715703 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 741/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 13:40:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 13:40:18.752818 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db5d5e0-2946-41cb-8a4f-03723c98e622 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:40:18.754744 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-4db5d5e0-2946-41cb-8a4f-03723c98e622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:40:18.857026 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db5d5e0-2946-41cb-8a4f-03723c98e622 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:40:18.922132 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:40:18.929096 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0f0ae9-c41d-40b8-a634-b7a785a3a892 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:40:18.929729 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0f0ae9-c41d-40b8-a634-b7a785a3a892 service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:40:18.930598 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0f0ae9-c41d-40b8-a634-b7a785a3a892 service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:40:18.930870 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0f0ae9-c41d-40b8-a634-b7a785a3a892 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:40:18.932455 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-0d0f0ae9-c41d-40b8-a634-b7a785a3a892 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:40:19.049240 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0f0ae9-c41d-40b8-a634-b7a785a3a892 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:40:19.050712 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0f0ae9-c41d-40b8-a634-b7a785a3a892 service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:40:19.068206 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 726/1467] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:40:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:40:21.057493 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e64c864-19d0-4d1e-a7f4-9d7524613ac3 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:40:21.057944 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e64c864-19d0-4d1e-a7f4-9d7524613ac3 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:40:21.058040 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e64c864-19d0-4d1e-a7f4-9d7524613ac3 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:40:21.059418 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 742/1468] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 13:40:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:40:21.064518 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751fc3fa-d9d6-469e-bdb1-f03179ac6e25 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:40:21.064952 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751fc3fa-d9d6-469e-bdb1-f03179ac6e25 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:40:21.065110 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751fc3fa-d9d6-469e-bdb1-f03179ac6e25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:40:21.130274 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-751fc3fa-d9d6-469e-bdb1-f03179ac6e25 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:40:21.132442 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-751fc3fa-d9d6-469e-bdb1-f03179ac6e25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:40:21.180219 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:40:21.229746 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 727/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 13:40:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:40:21.266021 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e64c864-19d0-4d1e-a7f4-9d7524613ac3 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:40:21.267583 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-8e64c864-19d0-4d1e-a7f4-9d7524613ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:40:21.371503 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e64c864-19d0-4d1e-a7f4-9d7524613ac3 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:40:21.419149 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:40:21.447470 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d0b875-e197-429f-8c74-e39297f980a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:40:21.448123 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d0b875-e197-429f-8c74-e39297f980a0 service neutron] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:40:21.448404 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d0b875-e197-429f-8c74-e39297f980a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:40:21.448676 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d0b875-e197-429f-8c74-e39297f980a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:40:21.450383 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-72d0b875-e197-429f-8c74-e39297f980a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:40:21.557823 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d0b875-e197-429f-8c74-e39297f980a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:40:21.559240 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d0b875-e197-429f-8c74-e39297f980a0 service neutron] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:40:21.577532 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 743/1470] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:40:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:40:31.227901 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=22 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:40:31.575058 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=66 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:40:40.918337 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd4a5eb-142c-41f4-afa1-a9c0549341f3 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:40:40.918905 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd4a5eb-142c-41f4-afa1-a9c0549341f3 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:40:40.918905 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd4a5eb-142c-41f4-afa1-a9c0549341f3 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:40:40.920242 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 728/1471] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:40:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:40:40.925497 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121adfd1-48a8-4ab5-beff-5bca4ff1c2b4 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:40:40.925934 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121adfd1-48a8-4ab5-beff-5bca4ff1c2b4 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:40:40.926046 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121adfd1-48a8-4ab5-beff-5bca4ff1c2b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:40:40.990546 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-121adfd1-48a8-4ab5-beff-5bca4ff1c2b4 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:40:40.991732 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-121adfd1-48a8-4ab5-beff-5bca4ff1c2b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:40:41.088245 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 744/1472] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:40:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:40:41.180820 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951407ad-f999-4544-abc8-40c70c09a4a8 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:40:41.181105 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951407ad-f999-4544-abc8-40c70c09a4a8 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:40:41.181385 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951407ad-f999-4544-abc8-40c70c09a4a8 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:40:41.182769 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 729/1473] 10.222.0.58 () {54 vars in 966 bytes} [Tue Jul 26 13:40:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:40:41.187505 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c8f189-c02a-41a3-ad45-6905d14d162a None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:40:41.187790 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c8f189-c02a-41a3-ad45-6905d14d162a None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:40:41.188066 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c8f189-c02a-41a3-ad45-6905d14d162a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:40:41.261978 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-c3c8f189-c02a-41a3-ad45-6905d14d162a None None] MFA Rules not processed for user `eaf065aef9a046b2adaec34816a336b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:40:41.263171 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-c3c8f189-c02a-41a3-ad45-6905d14d162a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:40:41.282830 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:40:41.338487 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 745/1474] 10.222.0.58 () {58 vars in 1063 bytes} [Tue Jul 26 13:40:41 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:40:41.345957 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951407ad-f999-4544-abc8-40c70c09a4a8 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:40:41.347470 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-951407ad-f999-4544-abc8-40c70c09a4a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:40:41.453953 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951407ad-f999-4544-abc8-40c70c09a4a8 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:40:41.525692 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d35a41-f846-455e-a8dd-cdb917f257c9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf065aef9a046b2adaec34816a336b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:40:41.526337 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d35a41-f846-455e-a8dd-cdb917f257c9 service glance] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:40:41.526619 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d35a41-f846-455e-a8dd-cdb917f257c9 service glance] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:40:41.526891 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d35a41-f846-455e-a8dd-cdb917f257c9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:40:41.528495 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-32d35a41-f846-455e-a8dd-cdb917f257c9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:40:41.581871 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:40:41.636212 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d35a41-f846-455e-a8dd-cdb917f257c9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:40:41.637662 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d35a41-f846-455e-a8dd-cdb917f257c9 service glance] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:40:41.655607 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 730/1475] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:40:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:40:43.748023 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff17d618-0c37-4d2f-a540-fff30aeb4dbd None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:40:43.748622 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff17d618-0c37-4d2f-a540-fff30aeb4dbd None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:40:43.748622 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff17d618-0c37-4d2f-a540-fff30aeb4dbd None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:40:43.750001 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 746/1476] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:40:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:40:43.755244 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd9e7bb-45b1-4464-9865-4dfdff808b19 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:40:43.755666 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd9e7bb-45b1-4464-9865-4dfdff808b19 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:40:43.755863 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd9e7bb-45b1-4464-9865-4dfdff808b19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:40:43.828390 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-4bd9e7bb-45b1-4464-9865-4dfdff808b19 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:40:43.829638 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-4bd9e7bb-45b1-4464-9865-4dfdff808b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:40:43.925412 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 731/1477] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:40:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:40:43.948632 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff17d618-0c37-4d2f-a540-fff30aeb4dbd None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:40:43.950248 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-ff17d618-0c37-4d2f-a540-fff30aeb4dbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:40:44.072891 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff17d618-0c37-4d2f-a540-fff30aeb4dbd None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:40:44.119906 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:40:44.164142 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9e3fbe-a453-469b-a6dd-1e1831a3a311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:40:44.164754 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9e3fbe-a453-469b-a6dd-1e1831a3a311 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:40:44.165030 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9e3fbe-a453-469b-a6dd-1e1831a3a311 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:40:44.165301 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9e3fbe-a453-469b-a6dd-1e1831a3a311 admin admin] PATH_INFO: `/v3/projects` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:40:44.166025 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9e3fbe-a453-469b-a6dd-1e1831a3a311 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:40:44.167743 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: WARNING py.warnings [None req-db9e3fbe-a453-469b-a6dd-1e1831a3a311 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 13:40:44.167743 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: warnings.warn(msg) Jul 26 13:40:44.167743 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: Jul 26 13:40:44.168862 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9e3fbe-a453-469b-a6dd-1e1831a3a311 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:40:44.187430 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 747/1478] 10.222.0.58 () {56 vars in 1108 bytes} [Tue Jul 26 13:40:43 2022] GET /identity/v3/projects => generated 2553 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:40:46.187193 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b340ffd9-b5b0-442d-b0c4-774752c57c97 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:40:46.187758 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b340ffd9-b5b0-442d-b0c4-774752c57c97 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:40:46.187758 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b340ffd9-b5b0-442d-b0c4-774752c57c97 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:40:46.189090 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 732/1479] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:40:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:40:46.194515 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad90dff-97bf-49c6-8a47-08574cdb0299 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:40:46.195016 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad90dff-97bf-49c6-8a47-08574cdb0299 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:40:46.195128 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad90dff-97bf-49c6-8a47-08574cdb0299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:40:46.259831 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-7ad90dff-97bf-49c6-8a47-08574cdb0299 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:40:46.261046 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-7ad90dff-97bf-49c6-8a47-08574cdb0299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:40:46.356224 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 748/1480] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:40:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:40:46.399406 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b340ffd9-b5b0-442d-b0c4-774752c57c97 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:40:46.400923 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-b340ffd9-b5b0-442d-b0c4-774752c57c97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:40:46.410662 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:40:46.545974 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b340ffd9-b5b0-442d-b0c4-774752c57c97 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:40:46.624230 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:40:46.656789 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c07ab2-7fd7-4318-93dc-ea39f95b6d33 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c69161cd520b4b29bf50f3bf2f1c033b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:40:46.657408 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c07ab2-7fd7-4318-93dc-ea39f95b6d33 service nova] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:40:46.657717 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c07ab2-7fd7-4318-93dc-ea39f95b6d33 service nova] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:40:46.657991 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c07ab2-7fd7-4318-93dc-ea39f95b6d33 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:40:46.659602 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-97c07ab2-7fd7-4318-93dc-ea39f95b6d33 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:40:46.747728 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c07ab2-7fd7-4318-93dc-ea39f95b6d33 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:40:46.749292 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c07ab2-7fd7-4318-93dc-ea39f95b6d33 service nova] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:40:46.767221 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 733/1481] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:40:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:40:48.935079 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d4a24f-122d-4029-8874-fd3cec82d455 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:40:48.935653 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d4a24f-122d-4029-8874-fd3cec82d455 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:40:48.935653 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d4a24f-122d-4029-8874-fd3cec82d455 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:40:48.937026 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 749/1482] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:40:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:40:48.942157 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7f7706-9591-42ad-8f86-e6bd1836a22e None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:40:48.942558 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7f7706-9591-42ad-8f86-e6bd1836a22e None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:40:48.942708 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7f7706-9591-42ad-8f86-e6bd1836a22e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:40:49.007271 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-9f7f7706-9591-42ad-8f86-e6bd1836a22e None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:40:49.008498 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-9f7f7706-9591-42ad-8f86-e6bd1836a22e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:40:49.056189 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:40:49.107301 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 734/1483] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:40:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:40:49.142654 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d4a24f-122d-4029-8874-fd3cec82d455 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:40:49.144214 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-36d4a24f-122d-4029-8874-fd3cec82d455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:40:49.168050 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:40:49.247893 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d4a24f-122d-4029-8874-fd3cec82d455 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:40:49.318984 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a35ad64-d3e1-45da-85e0-53b3fd461aab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:40:49.319623 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a35ad64-d3e1-45da-85e0-53b3fd461aab service neutron] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:40:49.320442 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a35ad64-d3e1-45da-85e0-53b3fd461aab service neutron] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:40:49.320714 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a35ad64-d3e1-45da-85e0-53b3fd461aab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:40:49.322370 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-3a35ad64-d3e1-45da-85e0-53b3fd461aab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:40:49.396936 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:40:49.432069 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a35ad64-d3e1-45da-85e0-53b3fd461aab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:40:49.433515 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a35ad64-d3e1-45da-85e0-53b3fd461aab service neutron] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:40:49.450908 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 750/1484] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:40:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:40:51.473461 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54df47b-8c16-4751-abae-09ca78abe37f None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:40:51.474047 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54df47b-8c16-4751-abae-09ca78abe37f None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:40:51.474047 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54df47b-8c16-4751-abae-09ca78abe37f None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:40:51.475377 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 735/1485] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:40:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:40:51.480516 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3be9f7-31b0-4dd3-bea3-476c4410150f None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:40:51.480936 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3be9f7-31b0-4dd3-bea3-476c4410150f None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:40:51.481074 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3be9f7-31b0-4dd3-bea3-476c4410150f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:40:51.547293 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-bc3be9f7-31b0-4dd3-bea3-476c4410150f None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:40:51.548502 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-bc3be9f7-31b0-4dd3-bea3-476c4410150f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:40:51.645017 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 751/1486] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:40:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:40:51.683894 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54df47b-8c16-4751-abae-09ca78abe37f None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:40:51.685495 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-e54df47b-8c16-4751-abae-09ca78abe37f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:40:51.788141 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54df47b-8c16-4751-abae-09ca78abe37f None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:40:51.857665 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:40:51.859613 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b92d362-d7d8-436a-aff2-b6f17a45db67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:40:51.860240 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b92d362-d7d8-436a-aff2-b6f17a45db67 service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:40:51.860722 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b92d362-d7d8-436a-aff2-b6f17a45db67 service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:40:51.861041 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b92d362-d7d8-436a-aff2-b6f17a45db67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:40:51.863139 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-9b92d362-d7d8-436a-aff2-b6f17a45db67 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:40:51.970989 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b92d362-d7d8-436a-aff2-b6f17a45db67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:40:51.972404 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b92d362-d7d8-436a-aff2-b6f17a45db67 service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:40:51.989953 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 736/1487] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:40:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:40:54.056884 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81125a7c-7fde-4eac-a171-955ec37f5982 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:40:54.057395 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81125a7c-7fde-4eac-a171-955ec37f5982 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:40:54.057547 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81125a7c-7fde-4eac-a171-955ec37f5982 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:40:54.058818 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 752/1488] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:40:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:40:54.063826 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22d62ff-efa2-4dd5-a800-8cfa1d172c53 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:40:54.064283 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22d62ff-efa2-4dd5-a800-8cfa1d172c53 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:40:54.064386 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22d62ff-efa2-4dd5-a800-8cfa1d172c53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:40:54.129468 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-a22d62ff-efa2-4dd5-a800-8cfa1d172c53 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:40:54.130697 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-a22d62ff-efa2-4dd5-a800-8cfa1d172c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:40:54.183635 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:40:54.227607 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 737/1489] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:40:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:40:54.269977 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81125a7c-7fde-4eac-a171-955ec37f5982 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:40:54.271625 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-81125a7c-7fde-4eac-a171-955ec37f5982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:40:54.308202 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:40:54.375120 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81125a7c-7fde-4eac-a171-955ec37f5982 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:40:54.445835 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421130a4-ebc5-49cd-a243-378440df9a09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:40:54.446449 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421130a4-ebc5-49cd-a243-378440df9a09 service neutron] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:40:54.446731 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421130a4-ebc5-49cd-a243-378440df9a09 service neutron] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:40:54.447004 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421130a4-ebc5-49cd-a243-378440df9a09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:40:54.448611 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-421130a4-ebc5-49cd-a243-378440df9a09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:40:54.525898 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:40:54.556571 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421130a4-ebc5-49cd-a243-378440df9a09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:40:54.558038 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421130a4-ebc5-49cd-a243-378440df9a09 service neutron] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:40:54.577387 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 753/1490] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:40:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:40:56.773401 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81928fe-986f-4b10-a7ce-95cd2cc34b91 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:40:56.774011 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81928fe-986f-4b10-a7ce-95cd2cc34b91 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:40:56.774011 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81928fe-986f-4b10-a7ce-95cd2cc34b91 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:40:56.775345 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 738/1491] 10.222.0.58 () {54 vars in 926 bytes} [Tue Jul 26 13:40:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:40:56.780589 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e24b9e-5b7c-41e4-a88c-aa846aae086d None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:40:56.780968 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e24b9e-5b7c-41e4-a88c-aa846aae086d None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:40:56.781191 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e24b9e-5b7c-41e4-a88c-aa846aae086d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:40:56.845606 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-c7e24b9e-5b7c-41e4-a88c-aa846aae086d None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:40:56.846803 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-c7e24b9e-5b7c-41e4-a88c-aa846aae086d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:40:56.941619 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 754/1492] 10.222.0.58 () {58 vars in 1023 bytes} [Tue Jul 26 13:40:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:40:56.980655 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81928fe-986f-4b10-a7ce-95cd2cc34b91 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:40:56.982233 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-f81928fe-986f-4b10-a7ce-95cd2cc34b91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:40:57.083502 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81928fe-986f-4b10-a7ce-95cd2cc34b91 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:40:57.154762 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf063253-2df0-4395-adad-6e3546be60ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:40:57.155380 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf063253-2df0-4395-adad-6e3546be60ef service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:40:57.155657 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf063253-2df0-4395-adad-6e3546be60ef service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:40:57.155929 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf063253-2df0-4395-adad-6e3546be60ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:40:57.157553 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-cf063253-2df0-4395-adad-6e3546be60ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:40:57.168444 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:40:57.275502 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf063253-2df0-4395-adad-6e3546be60ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:40:57.276916 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf063253-2df0-4395-adad-6e3546be60ef service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:40:57.295275 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 739/1493] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:40:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:41:06.939784 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=82 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:41:07.293012 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=50 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:42:24.217660 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b1a452-55c7-422d-9903-aecc658e890d None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:42:24.218239 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b1a452-55c7-422d-9903-aecc658e890d None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:42:24.218239 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b1a452-55c7-422d-9903-aecc658e890d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:42:24.296802 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-70b1a452-55c7-422d-9903-aecc658e890d None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:42:24.298182 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-70b1a452-55c7-422d-9903-aecc658e890d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:42:24.395859 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 755/1494] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:42:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:42:24.404350 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcefcb0d-8578-42fa-9ad9-e9ac8790cbe8 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:42:24.404849 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcefcb0d-8578-42fa-9ad9-e9ac8790cbe8 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:42:24.404954 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcefcb0d-8578-42fa-9ad9-e9ac8790cbe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:42:24.479360 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-dcefcb0d-8578-42fa-9ad9-e9ac8790cbe8 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:42:24.480572 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-dcefcb0d-8578-42fa-9ad9-e9ac8790cbe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:42:24.577890 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 740/1495] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:42:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:42:24.582484 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b1a452-55c7-422d-9903-aecc658e890d None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:42:24.583685 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-70b1a452-55c7-422d-9903-aecc658e890d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:42:24.595211 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b1a452-55c7-422d-9903-aecc658e890d None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:42:24.655910 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:42:24.689802 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b4d8a3-4a2e-4478-9d17-529e97dc53f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:42:24.690441 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b4d8a3-4a2e-4478-9d17-529e97dc53f6 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:42:24.690717 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b4d8a3-4a2e-4478-9d17-529e97dc53f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:42:24.690985 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b4d8a3-4a2e-4478-9d17-529e97dc53f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:42:24.691661 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b4d8a3-4a2e-4478-9d17-529e97dc53f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:42:24.691961 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b4d8a3-4a2e-4478-9d17-529e97dc53f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:42:24.692959 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: WARNING py.warnings [None req-e2b4d8a3-4a2e-4478-9d17-529e97dc53f6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 26 13:42:24.692959 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: warnings.warn(msg) Jul 26 13:42:24.692959 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: Jul 26 13:42:24.693858 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b4d8a3-4a2e-4478-9d17-529e97dc53f6 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:42:24.704564 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 756/1496] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Jul 26 13:42:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:42:24.712799 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcefcb0d-8578-42fa-9ad9-e9ac8790cbe8 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:42:24.722025 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcefcb0d-8578-42fa-9ad9-e9ac8790cbe8 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:42:24.783325 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:42:24.817223 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82657460-e4fa-42ab-8c0f-77000a5430c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:42:24.817879 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82657460-e4fa-42ab-8c0f-77000a5430c2 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:42:24.818162 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82657460-e4fa-42ab-8c0f-77000a5430c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:42:24.818433 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82657460-e4fa-42ab-8c0f-77000a5430c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:42:24.819628 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82657460-e4fa-42ab-8c0f-77000a5430c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:42:24.821354 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: WARNING py.warnings [None req-82657460-e4fa-42ab-8c0f-77000a5430c2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 13:42:24.821354 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: warnings.warn(msg) Jul 26 13:42:24.821354 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: Jul 26 13:42:24.822574 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82657460-e4fa-42ab-8c0f-77000a5430c2 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:42:24.876431 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 741/1497] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 13:42:24 2022] POST /identity/v3/projects => generated 377 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:42:24.880666 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b4d8a3-4a2e-4478-9d17-529e97dc53f6 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:42:24.882856 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-e2b4d8a3-4a2e-4478-9d17-529e97dc53f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:42:24.917137 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b4d8a3-4a2e-4478-9d17-529e97dc53f6 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:42:25.010085 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69236365-3b80-4212-8f73-a66ca78fe51f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:42:25.010703 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69236365-3b80-4212-8f73-a66ca78fe51f admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:42:25.011026 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69236365-3b80-4212-8f73-a66ca78fe51f admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:42:25.011296 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69236365-3b80-4212-8f73-a66ca78fe51f admin admin] PATH_INFO: `/v3/users` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:42:25.011974 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69236365-3b80-4212-8f73-a66ca78fe51f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:42:25.012970 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: WARNING py.warnings [None req-69236365-3b80-4212-8f73-a66ca78fe51f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 13:42:25.012970 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: warnings.warn(msg) Jul 26 13:42:25.012970 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: Jul 26 13:42:25.014200 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69236365-3b80-4212-8f73-a66ca78fe51f admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:42:25.057023 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 757/1498] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 13:42:24 2022] POST /identity/v3/users => generated 289 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 13:42:25.061160 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82657460-e4fa-42ab-8c0f-77000a5430c2 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:42:25.062720 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-82657460-e4fa-42ab-8c0f-77000a5430c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:42:25.125188 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82657460-e4fa-42ab-8c0f-77000a5430c2 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:42:25.208332 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:42:25.219991 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0883d5a-ca21-4541-a321-0ad691d239f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:42:25.220629 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0883d5a-ca21-4541-a321-0ad691d239f9 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:42:25.220911 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0883d5a-ca21-4541-a321-0ad691d239f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:42:25.221179 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0883d5a-ca21-4541-a321-0ad691d239f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:42:25.221902 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0883d5a-ca21-4541-a321-0ad691d239f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:42:25.223015 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: WARNING py.warnings [None req-b0883d5a-ca21-4541-a321-0ad691d239f9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 26 13:42:25.223015 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: warnings.warn(msg) Jul 26 13:42:25.223015 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: Jul 26 13:42:25.223898 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0883d5a-ca21-4541-a321-0ad691d239f9 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:42:25.236748 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 742/1499] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:42:25 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:42:25.240816 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69236365-3b80-4212-8f73-a66ca78fe51f admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:42:25.242403 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-69236365-3b80-4212-8f73-a66ca78fe51f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:42:25.253725 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69236365-3b80-4212-8f73-a66ca78fe51f admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:42:25.278479 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:42:25.348021 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46e58c9-965a-4df3-aad7-5c2bbe647adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:42:25.348643 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46e58c9-965a-4df3-aad7-5c2bbe647adf admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:42:25.348919 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46e58c9-965a-4df3-aad7-5c2bbe647adf admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:42:25.349187 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46e58c9-965a-4df3-aad7-5c2bbe647adf admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:42:25.349909 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46e58c9-965a-4df3-aad7-5c2bbe647adf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:42:25.351441 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: WARNING py.warnings [None req-e46e58c9-965a-4df3-aad7-5c2bbe647adf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 26 13:42:25.351441 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: warnings.warn(msg) Jul 26 13:42:25.351441 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: Jul 26 13:42:25.352310 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46e58c9-965a-4df3-aad7-5c2bbe647adf admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:42:25.364449 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 758/1500] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:42:25 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:42:25.368673 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0883d5a-ca21-4541-a321-0ad691d239f9 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:42:25.370202 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-b0883d5a-ca21-4541-a321-0ad691d239f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:42:25.402842 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0883d5a-ca21-4541-a321-0ad691d239f9 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:42:25.495977 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002e72e4-3883-4bce-9c1d-d4d116a49906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:42:25.496607 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002e72e4-3883-4bce-9c1d-d4d116a49906 admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:42:25.496831 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002e72e4-3883-4bce-9c1d-d4d116a49906 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:42:25.497103 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002e72e4-3883-4bce-9c1d-d4d116a49906 admin admin] PATH_INFO: `/v3/projects/a98768e68a934617920b13c183ac0df2/users/5fc94450857d4c35abe00a15564f3b6d/roles/759f389b221049c4b01faf3b984cd8d7` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:42:25.518787 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002e72e4-3883-4bce-9c1d-d4d116a49906 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a98768e68a934617920b13c183ac0df2, user_id=5fc94450857d4c35abe00a15564f3b6d, role_id=759f389b221049c4b01faf3b984cd8d7)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:42:25.519802 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: WARNING py.warnings [None req-002e72e4-3883-4bce-9c1d-d4d116a49906 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 13:42:25.519802 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: warnings.warn(msg) Jul 26 13:42:25.519802 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: Jul 26 13:42:25.521782 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002e72e4-3883-4bce-9c1d-d4d116a49906 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:42:25.545811 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 743/1501] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 13:42:25 2022] PUT /identity/v3/projects/a98768e68a934617920b13c183ac0df2/users/5fc94450857d4c35abe00a15564f3b6d/roles/759f389b221049c4b01faf3b984cd8d7 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:42:25.558001 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee3c4a1-9805-4f48-92c6-169ae3393639 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:42:25.558296 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee3c4a1-9805-4f48-92c6-169ae3393639 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:42:25.558578 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee3c4a1-9805-4f48-92c6-169ae3393639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:42:25.614600 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-8ee3c4a1-9805-4f48-92c6-169ae3393639 None None] MFA Rules not processed for user `5fc94450857d4c35abe00a15564f3b6d`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:42:25.615813 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-8ee3c4a1-9805-4f48-92c6-169ae3393639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:42:25.690973 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 759/1502] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:42:25 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:42:25.704555 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926b890b-17f7-43da-aa88-f8dd975f24d5 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:42:25.704847 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926b890b-17f7-43da-aa88-f8dd975f24d5 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:42:25.705129 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926b890b-17f7-43da-aa88-f8dd975f24d5 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:42:25.706549 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 744/1503] 10.222.0.58 () {54 vars in 964 bytes} [Tue Jul 26 13:42:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:42:25.711980 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a0a6c8-252f-4e84-9444-7696185bd045 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:42:25.712265 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a0a6c8-252f-4e84-9444-7696185bd045 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:42:25.712537 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a0a6c8-252f-4e84-9444-7696185bd045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:42:25.742698 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:42:25.789743 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-a8a0a6c8-252f-4e84-9444-7696185bd045 None None] MFA Rules not processed for user `72e400f267fd488d8439426a81ba0a25`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:42:25.790881 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-a8a0a6c8-252f-4e84-9444-7696185bd045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:42:25.863903 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 760/1504] 10.222.0.58 () {58 vars in 1061 bytes} [Tue Jul 26 13:42:25 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:42:25.872160 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926b890b-17f7-43da-aa88-f8dd975f24d5 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:42:25.873706 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-926b890b-17f7-43da-aa88-f8dd975f24d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:42:25.929352 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:42:25.983659 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926b890b-17f7-43da-aa88-f8dd975f24d5 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:42:26.057093 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da4bed9-023d-4156-b33b-68ef157593d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:42:26.057687 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da4bed9-023d-4156-b33b-68ef157593d1 service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:42:26.057976 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da4bed9-023d-4156-b33b-68ef157593d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:42:26.058256 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da4bed9-023d-4156-b33b-68ef157593d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:42:26.059899 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-5da4bed9-023d-4156-b33b-68ef157593d1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:42:26.161083 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da4bed9-023d-4156-b33b-68ef157593d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:42:26.162546 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da4bed9-023d-4156-b33b-68ef157593d1 service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:42:26.166591 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:42:26.181425 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 745/1505] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:42:25 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:42:26.227239 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f306fb8-c45a-4b9d-a152-b8701529a30a None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:42:26.227525 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f306fb8-c45a-4b9d-a152-b8701529a30a None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:42:26.227798 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f306fb8-c45a-4b9d-a152-b8701529a30a None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:42:26.229069 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 761/1506] 10.222.0.58 () {52 vars in 830 bytes} [Tue Jul 26 13:42:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:42:26.232728 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da4bed9-023d-4156-b33b-68ef157593d1 service cinder] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:42:26.233971 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-5da4bed9-023d-4156-b33b-68ef157593d1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:42:26.332338 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da4bed9-023d-4156-b33b-68ef157593d1 service cinder] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:42:26.427481 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f85f44-bb38-4ca0-999d-5a5e6b506152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:42:26.428125 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f85f44-bb38-4ca0-999d-5a5e6b506152 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:42:26.428973 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f85f44-bb38-4ca0-999d-5a5e6b506152 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:42:26.429234 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f85f44-bb38-4ca0-999d-5a5e6b506152 admin admin] PATH_INFO: `/v3/users/5fc94450857d4c35abe00a15564f3b6d` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:42:26.441645 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f85f44-bb38-4ca0-999d-5a5e6b506152 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fc94450857d4c35abe00a15564f3b6d)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:42:26.442665 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: WARNING py.warnings [None req-b0f85f44-bb38-4ca0-999d-5a5e6b506152 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 13:42:26.442665 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: warnings.warn(msg) Jul 26 13:42:26.442665 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: Jul 26 13:42:26.443877 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f85f44-bb38-4ca0-999d-5a5e6b506152 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:42:26.453517 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:42:26.507598 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-b0f85f44-bb38-4ca0-999d-5a5e6b506152 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fc94450857d4c35abe00a15564f3b6d'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:42:26.508263 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-b0f85f44-bb38-4ca0-999d-5a5e6b506152 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fc94450857d4c35abe00a15564f3b6d'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:42:26.520357 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-b0f85f44-bb38-4ca0-999d-5a5e6b506152 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fc94450857d4c35abe00a15564f3b6d'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:42:26.520901 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-b0f85f44-bb38-4ca0-999d-5a5e6b506152 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fc94450857d4c35abe00a15564f3b6d'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:42:26.541070 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-b0f85f44-bb38-4ca0-999d-5a5e6b506152 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fc94450857d4c35abe00a15564f3b6d'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:42:26.555500 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 746/1507] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 13:42:26 2022] DELETE /identity/v3/users/5fc94450857d4c35abe00a15564f3b6d => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:42:26.560012 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc44c486-04a6-4a64-bb1a-dbfbbe494685 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:42:26.560500 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc44c486-04a6-4a64-bb1a-dbfbbe494685 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:42:26.560626 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc44c486-04a6-4a64-bb1a-dbfbbe494685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:42:26.616596 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-bc44c486-04a6-4a64-bb1a-dbfbbe494685 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:42:26.617743 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-bc44c486-04a6-4a64-bb1a-dbfbbe494685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:42:26.691494 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:42:26.705267 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 762/1508] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:42:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 13:42:26.714724 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f85f44-bb38-4ca0-999d-5a5e6b506152 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:42:26.715944 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-b0f85f44-bb38-4ca0-999d-5a5e6b506152 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:42:26.814052 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f85f44-bb38-4ca0-999d-5a5e6b506152 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:42:26.869425 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:42:26.888981 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac3de2d-29f8-4206-8850-4a517540976e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:42:26.889635 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac3de2d-29f8-4206-8850-4a517540976e service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:42:26.889921 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac3de2d-29f8-4206-8850-4a517540976e service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:42:26.890209 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac3de2d-29f8-4206-8850-4a517540976e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:42:26.891819 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-3ac3de2d-29f8-4206-8850-4a517540976e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:42:27.000183 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac3de2d-29f8-4206-8850-4a517540976e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:42:27.001639 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac3de2d-29f8-4206-8850-4a517540976e service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:42:27.019424 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 747/1509] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:42:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:42:27.427368 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc44c486-04a6-4a64-bb1a-dbfbbe494685 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:42:27.428665 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-bc44c486-04a6-4a64-bb1a-dbfbbe494685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:42:27.440063 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc44c486-04a6-4a64-bb1a-dbfbbe494685 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:42:27.539802 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dafe69-57b2-422a-9762-a8dc0237d4e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:42:27.540460 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dafe69-57b2-422a-9762-a8dc0237d4e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:42:27.540741 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dafe69-57b2-422a-9762-a8dc0237d4e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:42:27.541012 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dafe69-57b2-422a-9762-a8dc0237d4e3 admin admin] PATH_INFO: `/v3/projects/a98768e68a934617920b13c183ac0df2` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:42:27.550986 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dafe69-57b2-422a-9762-a8dc0237d4e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a98768e68a934617920b13c183ac0df2)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:42:27.551999 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: WARNING py.warnings [None req-a2dafe69-57b2-422a-9762-a8dc0237d4e3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 13:42:27.551999 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: warnings.warn(msg) Jul 26 13:42:27.551999 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: Jul 26 13:42:27.553190 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dafe69-57b2-422a-9762-a8dc0237d4e3 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:42:27.611308 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-a2dafe69-57b2-422a-9762-a8dc0237d4e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a98768e68a934617920b13c183ac0df2', 'request_id': 'req-a2dafe69-57b2-422a-9762-a8dc0237d4e3'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:42:27.620865 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-a2dafe69-57b2-422a-9762-a8dc0237d4e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a98768e68a934617920b13c183ac0df2', 'request_id': 'req-a2dafe69-57b2-422a-9762-a8dc0237d4e3'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:42:27.626878 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-a2dafe69-57b2-422a-9762-a8dc0237d4e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a98768e68a934617920b13c183ac0df2', 'request_id': 'req-a2dafe69-57b2-422a-9762-a8dc0237d4e3'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:42:27.657859 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-a2dafe69-57b2-422a-9762-a8dc0237d4e3 admin admin] The token cache is being invalidate because project a98768e68a934617920b13c183ac0df2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86933) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 13:42:27.658147 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-a2dafe69-57b2-422a-9762-a8dc0237d4e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:42:27.658671 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-a2dafe69-57b2-422a-9762-a8dc0237d4e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:42:27.659988 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 763/1510] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 13:42:27 2022] DELETE /identity/v3/projects/a98768e68a934617920b13c183ac0df2 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:42:37.017108 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=62 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:42:37.655784 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:44:14.517845 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6021a7-a510-477e-ad3d-70115c9f2e88 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:44:14.518445 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6021a7-a510-477e-ad3d-70115c9f2e88 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:44:14.518445 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6021a7-a510-477e-ad3d-70115c9f2e88 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:44:14.519786 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 748/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 13:44:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:44:14.524860 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b7f5dc-7356-4e8b-9d1c-0fd71372426c None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:44:14.525327 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b7f5dc-7356-4e8b-9d1c-0fd71372426c None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:44:14.525434 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b7f5dc-7356-4e8b-9d1c-0fd71372426c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:44:14.605236 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-85b7f5dc-7356-4e8b-9d1c-0fd71372426c None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:44:14.606489 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-85b7f5dc-7356-4e8b-9d1c-0fd71372426c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:44:14.711570 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 764/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 13:44:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:44:14.864867 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6021a7-a510-477e-ad3d-70115c9f2e88 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:44:14.867195 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-2c6021a7-a510-477e-ad3d-70115c9f2e88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:44:14.984503 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6021a7-a510-477e-ad3d-70115c9f2e88 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:44:14.990494 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fde460-3e4c-4898-a046-2b2e8c087d6f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c69161cd520b4b29bf50f3bf2f1c033b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:44:14.991109 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fde460-3e4c-4898-a046-2b2e8c087d6f service nova] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:44:14.991393 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fde460-3e4c-4898-a046-2b2e8c087d6f service nova] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:44:14.991669 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fde460-3e4c-4898-a046-2b2e8c087d6f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:44:14.999585 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fde460-3e4c-4898-a046-2b2e8c087d6f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:44:15.001780 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fde460-3e4c-4898-a046-2b2e8c087d6f service nova] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:44:15.010970 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 749/1513] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:44:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:44:15.872517 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a8533-e272-4be9-adac-2e4e75f139a6 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:44:15.873067 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a8533-e272-4be9-adac-2e4e75f139a6 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:44:15.873067 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a8533-e272-4be9-adac-2e4e75f139a6 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:44:15.874506 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 765/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 13:44:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:44:15.879377 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51064384-3996-4d08-ad60-07afd2a60ae8 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:44:15.879914 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51064384-3996-4d08-ad60-07afd2a60ae8 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:44:15.879914 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51064384-3996-4d08-ad60-07afd2a60ae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:44:15.899377 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-51064384-3996-4d08-ad60-07afd2a60ae8 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:44:15.900556 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-51064384-3996-4d08-ad60-07afd2a60ae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:44:15.909149 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 750/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 13:44:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:44:16.051853 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa6c827-6f12-486e-8f21-f4a6d939a4ab None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:44:16.052141 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa6c827-6f12-486e-8f21-f4a6d939a4ab None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:44:16.052420 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa6c827-6f12-486e-8f21-f4a6d939a4ab None None] PATH_INFO: `/v3` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:44:16.053748 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 766/1516] 10.222.0.27 () {54 vars in 925 bytes} [Tue Jul 26 13:44:16 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:44:16.063160 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51064384-3996-4d08-ad60-07afd2a60ae8 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:44:16.066055 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcbd6c2-b58f-4117-9fee-f63a6b3d6a2d None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:44:16.066407 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcbd6c2-b58f-4117-9fee-f63a6b3d6a2d None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:44:16.066747 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcbd6c2-b58f-4117-9fee-f63a6b3d6a2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:44:16.075624 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51064384-3996-4d08-ad60-07afd2a60ae8 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:44:16.081705 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9ca047-4ed4-4a49-8b33-0843da8a5567 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c69161cd520b4b29bf50f3bf2f1c033b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:44:16.082318 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9ca047-4ed4-4a49-8b33-0843da8a5567 service nova] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:44:16.082600 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9ca047-4ed4-4a49-8b33-0843da8a5567 service nova] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:44:16.082873 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9ca047-4ed4-4a49-8b33-0843da8a5567 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:44:16.091115 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9ca047-4ed4-4a49-8b33-0843da8a5567 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:44:16.092683 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9ca047-4ed4-4a49-8b33-0843da8a5567 service nova] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:44:16.102693 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 751/1517] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:44:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:44:16.126404 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-6fcbd6c2-b58f-4117-9fee-f63a6b3d6a2d None None] MFA Rules not processed for user `835cbc13756b41b598142df3e35f5492`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:44:16.127533 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-6fcbd6c2-b58f-4117-9fee-f63a6b3d6a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:44:16.163703 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:44:16.208389 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 767/1518] 10.222.0.27 () {58 vars in 1015 bytes} [Tue Jul 26 13:44:16 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 13:44:20.029899 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59806c4-0c3b-4450-a157-b57968d91f66 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:44:20.030447 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59806c4-0c3b-4450-a157-b57968d91f66 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:44:20.030447 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59806c4-0c3b-4450-a157-b57968d91f66 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:44:20.031748 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 752/1519] 10.222.0.58 () {54 vars in 961 bytes} [Tue Jul 26 13:44:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:44:20.037673 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6324c6-5865-4dea-af6c-65b29fefd175 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:44:20.038160 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6324c6-5865-4dea-af6c-65b29fefd175 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:44:20.038266 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6324c6-5865-4dea-af6c-65b29fefd175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:44:20.057941 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-4d6324c6-5865-4dea-af6c-65b29fefd175 None None] MFA Rules not processed for user `835cbc13756b41b598142df3e35f5492`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:44:20.059130 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-4d6324c6-5865-4dea-af6c-65b29fefd175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:44:20.067729 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 768/1520] 10.222.0.58 () {58 vars in 1058 bytes} [Tue Jul 26 13:44:20 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:44:20.075302 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59806c4-0c3b-4450-a157-b57968d91f66 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:44:20.088450 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59806c4-0c3b-4450-a157-b57968d91f66 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:44:20.093339 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb14471-6231-4980-a7a4-5cb1f889cbd5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '835cbc13756b41b598142df3e35f5492', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:44:20.093965 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb14471-6231-4980-a7a4-5cb1f889cbd5 service placement] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:44:20.094248 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb14471-6231-4980-a7a4-5cb1f889cbd5 service placement] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:44:20.095072 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb14471-6231-4980-a7a4-5cb1f889cbd5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:44:20.103253 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb14471-6231-4980-a7a4-5cb1f889cbd5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:44:20.104805 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb14471-6231-4980-a7a4-5cb1f889cbd5 service placement] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:44:20.115048 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 753/1521] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:44:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:44:27.347887 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0799c08d-8092-4106-978b-3a0e24fb3b39 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:44:27.348440 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0799c08d-8092-4106-978b-3a0e24fb3b39 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:44:27.348440 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0799c08d-8092-4106-978b-3a0e24fb3b39 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:44:27.349931 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 769/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 13:44:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:44:27.354586 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c971c2c-8e54-49cb-af27-d396e5e17db1 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:44:27.354985 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c971c2c-8e54-49cb-af27-d396e5e17db1 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:44:27.355162 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c971c2c-8e54-49cb-af27-d396e5e17db1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:44:27.374531 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-9c971c2c-8e54-49cb-af27-d396e5e17db1 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:44:27.375708 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-9c971c2c-8e54-49cb-af27-d396e5e17db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:44:27.383339 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 754/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 13:44:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:44:27.535887 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0799c08d-8092-4106-978b-3a0e24fb3b39 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:44:27.550791 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0799c08d-8092-4106-978b-3a0e24fb3b39 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:44:27.556244 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a2c862-e14e-4ad1-931b-81b85654cc53 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c69161cd520b4b29bf50f3bf2f1c033b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:44:27.556848 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a2c862-e14e-4ad1-931b-81b85654cc53 service nova] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:44:27.557127 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a2c862-e14e-4ad1-931b-81b85654cc53 service nova] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:44:27.557399 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a2c862-e14e-4ad1-931b-81b85654cc53 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:44:27.565410 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a2c862-e14e-4ad1-931b-81b85654cc53 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:44:27.567011 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a2c862-e14e-4ad1-931b-81b85654cc53 service nova] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:44:27.576883 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 770/1524] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:44:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:44:28.389953 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f5435-7e27-4082-90b0-34c8ca9cc0b5 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:44:28.390506 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f5435-7e27-4082-90b0-34c8ca9cc0b5 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:44:28.390506 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f5435-7e27-4082-90b0-34c8ca9cc0b5 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:44:28.392599 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 755/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 13:44:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:44:28.397205 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0756634-87ae-4e3e-8f06-0a29cf395fb5 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:44:28.397927 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0756634-87ae-4e3e-8f06-0a29cf395fb5 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:44:28.397927 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0756634-87ae-4e3e-8f06-0a29cf395fb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:44:28.417322 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-c0756634-87ae-4e3e-8f06-0a29cf395fb5 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:44:28.418534 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-c0756634-87ae-4e3e-8f06-0a29cf395fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:44:28.426398 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 771/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 13:44:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:44:28.575959 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85f5435-7e27-4082-90b0-34c8ca9cc0b5 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:44:28.590157 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85f5435-7e27-4082-90b0-34c8ca9cc0b5 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:44:28.595588 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb72592-4bac-4604-846a-edf518534421 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c69161cd520b4b29bf50f3bf2f1c033b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:44:28.596188 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb72592-4bac-4604-846a-edf518534421 service nova] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:44:28.596476 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb72592-4bac-4604-846a-edf518534421 service nova] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:44:28.596752 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb72592-4bac-4604-846a-edf518534421 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:44:28.604650 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb72592-4bac-4604-846a-edf518534421 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:44:28.606257 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb72592-4bac-4604-846a-edf518534421 service nova] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:44:28.615973 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 756/1527] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:44:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 13:44:32.466435 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64c13da-3d41-4976-8c44-fbdc6925aa51 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:44:32.467008 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64c13da-3d41-4976-8c44-fbdc6925aa51 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:44:32.467008 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64c13da-3d41-4976-8c44-fbdc6925aa51 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:44:32.468406 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 772/1528] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 13:44:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:44:32.473439 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acf3bf1-7e08-409b-939a-2d975e6162d3 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:44:32.473897 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acf3bf1-7e08-409b-939a-2d975e6162d3 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:44:32.474005 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acf3bf1-7e08-409b-939a-2d975e6162d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:44:32.493532 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-5acf3bf1-7e08-409b-939a-2d975e6162d3 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:44:32.494724 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-5acf3bf1-7e08-409b-939a-2d975e6162d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:44:32.503051 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 757/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 13:44:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:44:32.576615 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64c13da-3d41-4976-8c44-fbdc6925aa51 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:44:32.578519 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-b64c13da-3d41-4976-8c44-fbdc6925aa51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:44:32.656410 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64c13da-3d41-4976-8c44-fbdc6925aa51 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:44:32.660711 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f130dc5-9a94-4d50-bd74-438ac614498a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:44:32.661319 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f130dc5-9a94-4d50-bd74-438ac614498a service neutron] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:44:32.661624 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f130dc5-9a94-4d50-bd74-438ac614498a service neutron] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:44:32.661894 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f130dc5-9a94-4d50-bd74-438ac614498a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:44:32.670446 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f130dc5-9a94-4d50-bd74-438ac614498a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:44:32.671846 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f130dc5-9a94-4d50-bd74-438ac614498a service neutron] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:44:32.681636 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 773/1530] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:44:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:44:34.655003 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8d391d-ba46-4759-9983-aecd75df95d0 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:44:34.655573 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8d391d-ba46-4759-9983-aecd75df95d0 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:44:34.655573 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8d391d-ba46-4759-9983-aecd75df95d0 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:44:34.656894 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 758/1531] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 13:44:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:44:34.663137 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f9d319-edde-44bd-bb68-421f2de17ffd None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:44:34.663528 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f9d319-edde-44bd-bb68-421f2de17ffd None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:44:34.663695 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f9d319-edde-44bd-bb68-421f2de17ffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:44:34.682442 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-f5f9d319-edde-44bd-bb68-421f2de17ffd None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:44:34.683633 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-f5f9d319-edde-44bd-bb68-421f2de17ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:44:34.691482 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 774/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 13:44:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:44:34.727991 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8d391d-ba46-4759-9983-aecd75df95d0 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:44:34.738550 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8d391d-ba46-4759-9983-aecd75df95d0 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:44:34.742008 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d89ce97-eaa4-4963-aed3-d187a0d51ff8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:44:34.742627 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d89ce97-eaa4-4963-aed3-d187a0d51ff8 service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:44:34.742907 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d89ce97-eaa4-4963-aed3-d187a0d51ff8 service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:44:34.743178 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d89ce97-eaa4-4963-aed3-d187a0d51ff8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:44:34.751819 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d89ce97-eaa4-4963-aed3-d187a0d51ff8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:44:34.753229 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d89ce97-eaa4-4963-aed3-d187a0d51ff8 service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:44:34.762530 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 759/1533] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:44:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:44:38.329460 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2654be99-d647-4edb-803e-f75c21e16fb2 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:44:38.330014 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2654be99-d647-4edb-803e-f75c21e16fb2 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:44:38.330014 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2654be99-d647-4edb-803e-f75c21e16fb2 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:44:38.331366 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 775/1534] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 13:44:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:44:38.336562 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc6b433-e35e-47e6-8af3-e5b8a8ebe251 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:44:38.336967 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc6b433-e35e-47e6-8af3-e5b8a8ebe251 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:44:38.337124 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc6b433-e35e-47e6-8af3-e5b8a8ebe251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:44:38.345277 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=87 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:44:38.357882 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-ddc6b433-e35e-47e6-8af3-e5b8a8ebe251 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:44:38.359072 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-ddc6b433-e35e-47e6-8af3-e5b8a8ebe251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:44:38.366738 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 760/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 13:44:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:44:38.669145 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2654be99-d647-4edb-803e-f75c21e16fb2 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:44:38.670823 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-2654be99-d647-4edb-803e-f75c21e16fb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:44:38.746221 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2654be99-d647-4edb-803e-f75c21e16fb2 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:44:38.749730 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adb89ae-659b-4b91-a25c-c7291cf3b81d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:44:38.750339 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adb89ae-659b-4b91-a25c-c7291cf3b81d service cinder] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:44:38.750620 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adb89ae-659b-4b91-a25c-c7291cf3b81d service cinder] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:44:38.751483 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adb89ae-659b-4b91-a25c-c7291cf3b81d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:44:38.758270 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:44:38.760654 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adb89ae-659b-4b91-a25c-c7291cf3b81d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:44:38.762379 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adb89ae-659b-4b91-a25c-c7291cf3b81d service cinder] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:44:38.772550 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 776/1536] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:44:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:44:40.787236 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eca650-2389-4450-9dc9-2837732f5804 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:44:40.787864 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eca650-2389-4450-9dc9-2837732f5804 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:44:40.787864 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eca650-2389-4450-9dc9-2837732f5804 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:44:40.789135 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 761/1537] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 13:44:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:44:40.794242 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1227491f-1383-49a0-aaea-0dd1989107f4 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:44:40.794683 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1227491f-1383-49a0-aaea-0dd1989107f4 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:44:40.794791 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1227491f-1383-49a0-aaea-0dd1989107f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:44:40.814438 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-1227491f-1383-49a0-aaea-0dd1989107f4 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:44:40.815625 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-1227491f-1383-49a0-aaea-0dd1989107f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:44:40.823340 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 777/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 13:44:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 13:44:41.123321 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53eca650-2389-4450-9dc9-2837732f5804 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:44:41.124999 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-53eca650-2389-4450-9dc9-2837732f5804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:44:41.138174 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53eca650-2389-4450-9dc9-2837732f5804 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:44:41.141767 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e357ec-b973-4716-87ea-1ac02e860cc7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:44:41.142403 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e357ec-b973-4716-87ea-1ac02e860cc7 service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:44:41.143172 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e357ec-b973-4716-87ea-1ac02e860cc7 service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:44:41.143442 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e357ec-b973-4716-87ea-1ac02e860cc7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:44:41.152208 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e357ec-b973-4716-87ea-1ac02e860cc7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:44:41.153647 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e357ec-b973-4716-87ea-1ac02e860cc7 service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:44:41.163022 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 762/1539] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:44:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:44:50.809187 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=5 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:44:51.159087 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=8 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:12.450000 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cd5dd9-b144-4e1e-b121-094b5dac6e0d None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:12.453329 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cd5dd9-b144-4e1e-b121-094b5dac6e0d None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:12.453329 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cd5dd9-b144-4e1e-b121-094b5dac6e0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:12.478316 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-a0cd5dd9-b144-4e1e-b121-094b5dac6e0d None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:12.479539 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-a0cd5dd9-b144-4e1e-b121-094b5dac6e0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:12.494122 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 778/1540] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:46:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 13:46:12.503060 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fd8f67-012d-4849-b007-4b076f1136c0 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:12.507199 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fd8f67-012d-4849-b007-4b076f1136c0 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:12.507835 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fd8f67-012d-4849-b007-4b076f1136c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:12.538327 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-36fd8f67-012d-4849-b007-4b076f1136c0 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:12.539924 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-36fd8f67-012d-4849-b007-4b076f1136c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:12.549339 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 763/1541] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:46:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:46:12.554430 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cd5dd9-b144-4e1e-b121-094b5dac6e0d None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:12.569511 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cd5dd9-b144-4e1e-b121-094b5dac6e0d None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:12.574959 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c071e56a-01cb-43c1-a5e7-5a31af76a2ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:12.575646 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c071e56a-01cb-43c1-a5e7-5a31af76a2ea admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:12.575929 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c071e56a-01cb-43c1-a5e7-5a31af76a2ea admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:12.576208 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c071e56a-01cb-43c1-a5e7-5a31af76a2ea admin admin] PATH_INFO: `/v3/domains` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:12.576935 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c071e56a-01cb-43c1-a5e7-5a31af76a2ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:46:12.577246 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c071e56a-01cb-43c1-a5e7-5a31af76a2ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:12.578904 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c071e56a-01cb-43c1-a5e7-5a31af76a2ea admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:12.605430 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 779/1542] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Jul 26 13:46:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:46:12.614701 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fd8f67-012d-4849-b007-4b076f1136c0 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:12.635116 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fd8f67-012d-4849-b007-4b076f1136c0 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:12.644766 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1f4223-1666-4156-9e08-0309dc48c71e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:12.645415 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1f4223-1666-4156-9e08-0309dc48c71e admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:12.645976 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1f4223-1666-4156-9e08-0309dc48c71e admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:12.646266 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1f4223-1666-4156-9e08-0309dc48c71e admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:12.646975 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1f4223-1666-4156-9e08-0309dc48c71e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:12.649286 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1f4223-1666-4156-9e08-0309dc48c71e admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:12.722527 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 764/1543] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 13:46:12 2022] POST /identity/v3/projects => generated 371 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:46:12.724656 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c071e56a-01cb-43c1-a5e7-5a31af76a2ea admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:12.744532 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c071e56a-01cb-43c1-a5e7-5a31af76a2ea admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:12.819846 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5794e8-a526-442c-8309-50b11a139fcd None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:12.820211 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5794e8-a526-442c-8309-50b11a139fcd None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:12.820814 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5794e8-a526-442c-8309-50b11a139fcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:12.822801 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ade8d1a-7ae9-44b9-bd5c-50db4e5f189e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:12.823787 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ade8d1a-7ae9-44b9-bd5c-50db4e5f189e admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:12.824287 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ade8d1a-7ae9-44b9-bd5c-50db4e5f189e admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:12.824775 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ade8d1a-7ae9-44b9-bd5c-50db4e5f189e admin admin] PATH_INFO: `/v3/users` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:12.827917 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ade8d1a-7ae9-44b9-bd5c-50db4e5f189e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:12.831524 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ade8d1a-7ae9-44b9-bd5c-50db4e5f189e admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:12.870655 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-9f5794e8-a526-442c-8309-50b11a139fcd None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:12.872124 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-9f5794e8-a526-442c-8309-50b11a139fcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:12.873602 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 780/1544] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 13:46:12 2022] POST /identity/v3/users => generated 286 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:46:12.877938 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ade8d1a-7ae9-44b9-bd5c-50db4e5f189e admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:12.886868 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 765/1545] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:46:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:46:12.894792 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ade8d1a-7ae9-44b9-bd5c-50db4e5f189e admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:12.896699 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e2d8a-747e-4ff2-85d3-38cc8c4d6085 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:12.897029 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e2d8a-747e-4ff2-85d3-38cc8c4d6085 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:12.897308 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e2d8a-747e-4ff2-85d3-38cc8c4d6085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:12.904762 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aab7f2-5372-4972-bb39-732fc4c5d743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:12.905690 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aab7f2-5372-4972-bb39-732fc4c5d743 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:12.906461 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aab7f2-5372-4972-bb39-732fc4c5d743 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:12.906938 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aab7f2-5372-4972-bb39-732fc4c5d743 admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:12.907930 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aab7f2-5372-4972-bb39-732fc4c5d743 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:12.910111 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aab7f2-5372-4972-bb39-732fc4c5d743 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:12.930199 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 781/1546] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:46:12 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 13:46:12.931580 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-dc7e2d8a-747e-4ff2-85d3-38cc8c4d6085 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:12.932798 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-dc7e2d8a-747e-4ff2-85d3-38cc8c4d6085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:12.933946 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aab7f2-5372-4972-bb39-732fc4c5d743 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:12.951471 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aab7f2-5372-4972-bb39-732fc4c5d743 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:12.959581 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 766/1547] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:46:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:46:12.961126 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f67e5a-cb28-4148-bce4-db968ef0d895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:12.962033 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f67e5a-cb28-4148-bce4-db968ef0d895 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:12.962546 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f67e5a-cb28-4148-bce4-db968ef0d895 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:12.963062 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f67e5a-cb28-4148-bce4-db968ef0d895 admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:12.963853 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7e2d8a-747e-4ff2-85d3-38cc8c4d6085 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:12.964211 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f67e5a-cb28-4148-bce4-db968ef0d895 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:12.966033 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f67e5a-cb28-4148-bce4-db968ef0d895 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:12.983325 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 782/1548] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:46:12 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:46:12.987639 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f67e5a-cb28-4148-bce4-db968ef0d895 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:12.988918 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7e2d8a-747e-4ff2-85d3-38cc8c4d6085 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:13.005110 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f67e5a-cb28-4148-bce4-db968ef0d895 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:13.018350 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cac1fb1-990d-457f-b05a-16d2ba194a15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:13.018741 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701a3696-ac4b-4604-a562-3f0202352787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:13.019004 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cac1fb1-990d-457f-b05a-16d2ba194a15 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:13.019445 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cac1fb1-990d-457f-b05a-16d2ba194a15 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:13.019716 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cac1fb1-990d-457f-b05a-16d2ba194a15 admin admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:13.020359 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701a3696-ac4b-4604-a562-3f0202352787 admin admin] REQUEST_METHOD: `PUT` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:13.020481 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cac1fb1-990d-457f-b05a-16d2ba194a15 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:46:13.020712 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cac1fb1-990d-457f-b05a-16d2ba194a15 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:13.021302 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701a3696-ac4b-4604-a562-3f0202352787 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:13.022086 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701a3696-ac4b-4604-a562-3f0202352787 admin admin] PATH_INFO: `/v3/projects/2842eeced6c14c8dbe84359801ac5fdb/users/23b1773e3f4e47aca50c664edf55882e/roles/759f389b221049c4b01faf3b984cd8d7` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:13.025517 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: WARNING py.warnings [None req-2cac1fb1-990d-457f-b05a-16d2ba194a15 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 26 13:46:13.025517 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: warnings.warn(msg) Jul 26 13:46:13.025517 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: Jul 26 13:46:13.025517 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cac1fb1-990d-457f-b05a-16d2ba194a15 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:13.055921 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 767/1549] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Jul 26 13:46:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:46:13.057650 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e3800d-b7a0-46c5-89c8-8ae91deb4a47 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:13.058115 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e3800d-b7a0-46c5-89c8-8ae91deb4a47 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:13.058556 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e3800d-b7a0-46c5-89c8-8ae91deb4a47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:13.073565 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701a3696-ac4b-4604-a562-3f0202352787 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2842eeced6c14c8dbe84359801ac5fdb, user_id=23b1773e3f4e47aca50c664edf55882e, role_id=759f389b221049c4b01faf3b984cd8d7)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:13.073565 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: WARNING py.warnings [None req-701a3696-ac4b-4604-a562-3f0202352787 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 13:46:13.073565 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: warnings.warn(msg) Jul 26 13:46:13.073565 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: Jul 26 13:46:13.075306 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701a3696-ac4b-4604-a562-3f0202352787 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:13.097267 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-19e3800d-b7a0-46c5-89c8-8ae91deb4a47 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:13.099163 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-19e3800d-b7a0-46c5-89c8-8ae91deb4a47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:13.116160 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 783/1550] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 13:46:12 2022] PUT /identity/v3/projects/2842eeced6c14c8dbe84359801ac5fdb/users/23b1773e3f4e47aca50c664edf55882e/roles/759f389b221049c4b01faf3b984cd8d7 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 13:46:13.117130 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701a3696-ac4b-4604-a562-3f0202352787 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:13.259815 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701a3696-ac4b-4604-a562-3f0202352787 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:13.270540 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9375594-46a5-4fab-afef-40c5f308e0e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:13.271212 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9375594-46a5-4fab-afef-40c5f308e0e8 admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:13.271776 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9375594-46a5-4fab-afef-40c5f308e0e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:13.272671 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9375594-46a5-4fab-afef-40c5f308e0e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:13.272871 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9375594-46a5-4fab-afef-40c5f308e0e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:13.274315 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: WARNING py.warnings [None req-e9375594-46a5-4fab-afef-40c5f308e0e8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 13:46:13.274315 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: warnings.warn(msg) Jul 26 13:46:13.274315 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: Jul 26 13:46:13.275809 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9375594-46a5-4fab-afef-40c5f308e0e8 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:13.301376 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 768/1551] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:46:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:46:13.302869 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4465c3db-d8dd-4785-a383-3c9cac2165db None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:13.303169 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4465c3db-d8dd-4785-a383-3c9cac2165db None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:13.303442 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4465c3db-d8dd-4785-a383-3c9cac2165db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:13.314067 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 784/1552] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 13:46:13 2022] POST /identity/v3/projects => generated 387 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:46:13.316365 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37caf3f5-0777-4542-8415-a18f57433a78 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:13.316721 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37caf3f5-0777-4542-8415-a18f57433a78 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:13.317070 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37caf3f5-0777-4542-8415-a18f57433a78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:13.336199 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-4465c3db-d8dd-4785-a383-3c9cac2165db None None] MFA Rules not processed for user `23b1773e3f4e47aca50c664edf55882e`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:13.337418 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-4465c3db-d8dd-4785-a383-3c9cac2165db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:13.339844 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-37caf3f5-0777-4542-8415-a18f57433a78 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:13.341069 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-37caf3f5-0777-4542-8415-a18f57433a78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:13.345990 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:13.404234 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 769/1553] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:46:13 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:46:13.405524 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e51cbe-fda8-4a54-82a2-c1d7e0211fd9 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:13.405823 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e51cbe-fda8-4a54-82a2-c1d7e0211fd9 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:13.406105 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e51cbe-fda8-4a54-82a2-c1d7e0211fd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:13.414851 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 785/1554] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:46:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:46:13.415978 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11038311-63db-4780-8051-6d1453ee111b None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:13.416252 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11038311-63db-4780-8051-6d1453ee111b None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:13.416533 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11038311-63db-4780-8051-6d1453ee111b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:13.434108 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-55e51cbe-fda8-4a54-82a2-c1d7e0211fd9 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:13.435571 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-55e51cbe-fda8-4a54-82a2-c1d7e0211fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:13.437795 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-11038311-63db-4780-8051-6d1453ee111b None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:13.439102 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-11038311-63db-4780-8051-6d1453ee111b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:13.446529 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 770/1555] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:46:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:46:13.447848 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07db863c-c2f8-4091-9fd7-e37bd8900022 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:13.448171 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07db863c-c2f8-4091-9fd7-e37bd8900022 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:13.448509 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07db863c-c2f8-4091-9fd7-e37bd8900022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:13.449650 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 786/1556] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:46:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:46:13.450914 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15df0d31-83b5-415c-a242-f1b367aa2b7f None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:13.451488 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15df0d31-83b5-415c-a242-f1b367aa2b7f None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:13.451488 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15df0d31-83b5-415c-a242-f1b367aa2b7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:13.478148 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-15df0d31-83b5-415c-a242-f1b367aa2b7f None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:13.479393 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-15df0d31-83b5-415c-a242-f1b367aa2b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:13.479921 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-07db863c-c2f8-4091-9fd7-e37bd8900022 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:13.481143 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-07db863c-c2f8-4091-9fd7-e37bd8900022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:13.489262 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 771/1557] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:46:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:46:13.490656 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060082c7-5f71-4d33-8065-0718c927c197 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:13.491017 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060082c7-5f71-4d33-8065-0718c927c197 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:13.491364 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060082c7-5f71-4d33-8065-0718c927c197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:13.492149 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 787/1558] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:46:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:46:13.500102 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15df0d31-83b5-415c-a242-f1b367aa2b7f None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:13.510604 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15df0d31-83b5-415c-a242-f1b367aa2b7f None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:13.513161 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-060082c7-5f71-4d33-8065-0718c927c197 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:13.514380 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-060082c7-5f71-4d33-8065-0718c927c197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:13.515163 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8115b9-695a-408f-b5b6-5899fb32f31d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:13.515852 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8115b9-695a-408f-b5b6-5899fb32f31d admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:13.516137 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8115b9-695a-408f-b5b6-5899fb32f31d admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:13.516407 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8115b9-695a-408f-b5b6-5899fb32f31d admin admin] PATH_INFO: `/v3/users` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:13.517063 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8115b9-695a-408f-b5b6-5899fb32f31d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:13.519554 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8115b9-695a-408f-b5b6-5899fb32f31d admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:13.522774 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 772/1559] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:46:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:46:13.524024 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b4a9ee-ca39-4c57-ba93-23395e1fb5e5 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:13.524315 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b4a9ee-ca39-4c57-ba93-23395e1fb5e5 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:13.524595 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b4a9ee-ca39-4c57-ba93-23395e1fb5e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:13.539576 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:13.540968 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 788/1560] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 13:46:13 2022] POST /identity/v3/users => generated 294 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:46:13.542138 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ad53d0-d091-4f63-9554-88e34a4f00d8 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:13.542430 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ad53d0-d091-4f63-9554-88e34a4f00d8 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:13.542704 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ad53d0-d091-4f63-9554-88e34a4f00d8 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:13.544060 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 789/1561] 10.222.0.58 () {54 vars in 962 bytes} [Tue Jul 26 13:46:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:46:13.545049 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2546fd2-ff03-4f7f-a779-17e20ca5c234 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:13.545334 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2546fd2-ff03-4f7f-a779-17e20ca5c234 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:13.545652 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-58b4a9ee-ca39-4c57-ba93-23395e1fb5e5 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:13.545799 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2546fd2-ff03-4f7f-a779-17e20ca5c234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:13.547965 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-58b4a9ee-ca39-4c57-ba93-23395e1fb5e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:13.556575 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 773/1562] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:46:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:46:13.557873 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714f6e55-93cb-4f40-b66e-fea8d955a764 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:13.558162 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714f6e55-93cb-4f40-b66e-fea8d955a764 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:13.558437 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714f6e55-93cb-4f40-b66e-fea8d955a764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:13.566112 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-b2546fd2-ff03-4f7f-a779-17e20ca5c234 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:13.567299 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-b2546fd2-ff03-4f7f-a779-17e20ca5c234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:13.575117 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 790/1563] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:46:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:46:13.576281 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fb7bb4-aea4-4fca-bc63-89ab71a72f85 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:13.576562 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fb7bb4-aea4-4fca-bc63-89ab71a72f85 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:13.576836 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fb7bb4-aea4-4fca-bc63-89ab71a72f85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:13.578098 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-714f6e55-93cb-4f40-b66e-fea8d955a764 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:13.579388 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-714f6e55-93cb-4f40-b66e-fea8d955a764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:13.587863 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 774/1564] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:46:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:46:13.589132 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a717d6-ff91-40a6-a681-ed1de4c4c626 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:13.589424 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a717d6-ff91-40a6-a681-ed1de4c4c626 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:13.589733 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a717d6-ff91-40a6-a681-ed1de4c4c626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:13.599850 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-64fb7bb4-aea4-4fca-bc63-89ab71a72f85 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:13.601272 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-64fb7bb4-aea4-4fca-bc63-89ab71a72f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:13.608160 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-09a717d6-ff91-40a6-a681-ed1de4c4c626 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:13.609323 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-09a717d6-ff91-40a6-a681-ed1de4c4c626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:13.609492 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 791/1565] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:46:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:46:13.613319 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fb7bb4-aea4-4fca-bc63-89ab71a72f85 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:13.617674 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 775/1566] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:46:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:46:13.618327 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a717d6-ff91-40a6-a681-ed1de4c4c626 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:13.622783 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fb7bb4-aea4-4fca-bc63-89ab71a72f85 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:13.627887 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a717d6-ff91-40a6-a681-ed1de4c4c626 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:13.628034 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3162261-bc81-4a32-9a21-754ab1380322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:13.628539 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3162261-bc81-4a32-9a21-754ab1380322 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:13.628822 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3162261-bc81-4a32-9a21-754ab1380322 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:13.629098 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3162261-bc81-4a32-9a21-754ab1380322 admin admin] PATH_INFO: `/v3/domains` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:13.629840 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3162261-bc81-4a32-9a21-754ab1380322 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:46:13.630254 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3162261-bc81-4a32-9a21-754ab1380322 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:13.631846 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3162261-bc81-4a32-9a21-754ab1380322 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:13.632552 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1194728e-3e91-4a90-aaa0-1b6d80145af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:13.633137 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1194728e-3e91-4a90-aaa0-1b6d80145af8 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:13.633422 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1194728e-3e91-4a90-aaa0-1b6d80145af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:13.633739 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1194728e-3e91-4a90-aaa0-1b6d80145af8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:13.634417 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1194728e-3e91-4a90-aaa0-1b6d80145af8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:13.635982 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1194728e-3e91-4a90-aaa0-1b6d80145af8 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:13.644461 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 792/1567] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Jul 26 13:46:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:46:13.645672 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68354c37-9c89-48e5-8352-07d474dcb6b3 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:13.645962 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68354c37-9c89-48e5-8352-07d474dcb6b3 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:13.646250 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68354c37-9c89-48e5-8352-07d474dcb6b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:13.646352 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 776/1568] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:46:13 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:46:13.647139 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1194728e-3e91-4a90-aaa0-1b6d80145af8 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:13.663777 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1194728e-3e91-4a90-aaa0-1b6d80145af8 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:13.668407 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac3ca80-1589-4059-96e5-bfa8d6ba2845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:13.669004 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac3ca80-1589-4059-96e5-bfa8d6ba2845 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:13.669285 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac3ca80-1589-4059-96e5-bfa8d6ba2845 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:13.669619 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac3ca80-1589-4059-96e5-bfa8d6ba2845 admin admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:13.670312 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac3ca80-1589-4059-96e5-bfa8d6ba2845 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:46:13.670642 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac3ca80-1589-4059-96e5-bfa8d6ba2845 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:13.672286 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac3ca80-1589-4059-96e5-bfa8d6ba2845 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:13.684231 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-68354c37-9c89-48e5-8352-07d474dcb6b3 None None] MFA Rules not processed for user `c69161cd520b4b29bf50f3bf2f1c033b`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:13.685315 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 777/1569] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Jul 26 13:46:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:46:13.685574 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-68354c37-9c89-48e5-8352-07d474dcb6b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:13.685976 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac3ca80-1589-4059-96e5-bfa8d6ba2845 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:13.696431 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac3ca80-1589-4059-96e5-bfa8d6ba2845 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:13.701309 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bdc970-6a71-4bbf-b165-45bd775a4936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:13.701992 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bdc970-6a71-4bbf-b165-45bd775a4936 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:13.702268 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bdc970-6a71-4bbf-b165-45bd775a4936 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:13.702538 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bdc970-6a71-4bbf-b165-45bd775a4936 admin admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:13.703217 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bdc970-6a71-4bbf-b165-45bd775a4936 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:46:13.703516 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bdc970-6a71-4bbf-b165-45bd775a4936 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:13.705666 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bdc970-6a71-4bbf-b165-45bd775a4936 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:13.717045 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 778/1570] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Jul 26 13:46:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:46:13.723157 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bdc970-6a71-4bbf-b165-45bd775a4936 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:13.732484 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bdc970-6a71-4bbf-b165-45bd775a4936 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:13.737459 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553fa532-9775-4e70-92d6-f25f7815e458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:13.738063 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553fa532-9775-4e70-92d6-f25f7815e458 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:13.738343 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553fa532-9775-4e70-92d6-f25f7815e458 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:13.738614 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553fa532-9775-4e70-92d6-f25f7815e458 admin admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:13.739286 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553fa532-9775-4e70-92d6-f25f7815e458 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:46:13.739588 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553fa532-9775-4e70-92d6-f25f7815e458 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:13.741130 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553fa532-9775-4e70-92d6-f25f7815e458 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:13.753878 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 779/1571] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Jul 26 13:46:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:46:13.754990 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553fa532-9775-4e70-92d6-f25f7815e458 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:13.755751 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 793/1572] 10.222.0.58 () {58 vars in 1059 bytes} [Tue Jul 26 13:46:13 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:46:13.756569 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68354c37-9c89-48e5-8352-07d474dcb6b3 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:13.770043 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553fa532-9775-4e70-92d6-f25f7815e458 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:13.771305 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68354c37-9c89-48e5-8352-07d474dcb6b3 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:13.774636 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9af1241-2799-416f-8489-aaa976aeff4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:13.775242 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9af1241-2799-416f-8489-aaa976aeff4e admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:13.775528 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9af1241-2799-416f-8489-aaa976aeff4e admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:13.775790 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9af1241-2799-416f-8489-aaa976aeff4e admin admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:13.776052 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae252c2-d4cf-4e38-9e90-b532a2072d35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:13.776503 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9af1241-2799-416f-8489-aaa976aeff4e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:46:13.776687 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae252c2-d4cf-4e38-9e90-b532a2072d35 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:13.776796 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9af1241-2799-416f-8489-aaa976aeff4e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:13.776960 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae252c2-d4cf-4e38-9e90-b532a2072d35 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:13.777193 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae252c2-d4cf-4e38-9e90-b532a2072d35 admin admin] PATH_INFO: `/v3/domains` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:13.777918 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae252c2-d4cf-4e38-9e90-b532a2072d35 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:46:13.778221 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae252c2-d4cf-4e38-9e90-b532a2072d35 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:13.778401 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9af1241-2799-416f-8489-aaa976aeff4e admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:13.779774 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae252c2-d4cf-4e38-9e90-b532a2072d35 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:13.791684 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 780/1573] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Jul 26 13:46:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:46:13.792782 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9af1241-2799-416f-8489-aaa976aeff4e admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:13.800770 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 794/1574] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Jul 26 13:46:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:46:13.800770 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae252c2-d4cf-4e38-9e90-b532a2072d35 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:13.804915 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae252c2-d4cf-4e38-9e90-b532a2072d35 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:13.805916 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9af1241-2799-416f-8489-aaa976aeff4e admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:13.809584 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2cf96e-dad4-4892-a167-87c651f0b6d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:13.810192 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2cf96e-dad4-4892-a167-87c651f0b6d9 admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:13.810409 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691aafaa-5193-4b39-b5b2-dd3a928c5577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:13.810543 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2cf96e-dad4-4892-a167-87c651f0b6d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:13.810744 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2cf96e-dad4-4892-a167-87c651f0b6d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:13.811001 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691aafaa-5193-4b39-b5b2-dd3a928c5577 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:13.811279 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691aafaa-5193-4b39-b5b2-dd3a928c5577 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:13.811408 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2cf96e-dad4-4892-a167-87c651f0b6d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:13.811566 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691aafaa-5193-4b39-b5b2-dd3a928c5577 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:13.812236 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691aafaa-5193-4b39-b5b2-dd3a928c5577 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:13.813300 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2cf96e-dad4-4892-a167-87c651f0b6d9 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:13.813824 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691aafaa-5193-4b39-b5b2-dd3a928c5577 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:13.824758 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 781/1575] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:46:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:46:13.825527 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691aafaa-5193-4b39-b5b2-dd3a928c5577 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:13.829318 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:13.836521 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691aafaa-5193-4b39-b5b2-dd3a928c5577 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:13.841345 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b34c3f3-e9b2-4ae4-bc59-b716bbb05ab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:13.841965 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b34c3f3-e9b2-4ae4-bc59-b716bbb05ab1 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:13.842263 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b34c3f3-e9b2-4ae4-bc59-b716bbb05ab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:13.842516 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b34c3f3-e9b2-4ae4-bc59-b716bbb05ab1 admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:13.843169 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b34c3f3-e9b2-4ae4-bc59-b716bbb05ab1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:13.847445 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 795/1576] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 13:46:13 2022] POST /identity/v3/projects => generated 387 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:46:13.847445 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2cf96e-dad4-4892-a167-87c651f0b6d9 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:13.847678 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b34c3f3-e9b2-4ae4-bc59-b716bbb05ab1 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:13.872872 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 782/1577] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 13:46:13 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:46:13.873953 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b34c3f3-e9b2-4ae4-bc59-b716bbb05ab1 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:13.909361 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2cf96e-dad4-4892-a167-87c651f0b6d9 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:13.937559 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b34c3f3-e9b2-4ae4-bc59-b716bbb05ab1 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:13.943040 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2517ae7-c282-4110-98e0-e4e2be4a5bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:13.943732 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2517ae7-c282-4110-98e0-e4e2be4a5bb1 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:13.944154 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2517ae7-c282-4110-98e0-e4e2be4a5bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:13.944440 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2517ae7-c282-4110-98e0-e4e2be4a5bb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:13.945129 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2517ae7-c282-4110-98e0-e4e2be4a5bb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:13.947120 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2517ae7-c282-4110-98e0-e4e2be4a5bb1 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:13.965596 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4db218-0af2-4b76-934a-3ea1426b0bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:13.966277 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4db218-0af2-4b76-934a-3ea1426b0bd2 admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:13.966564 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4db218-0af2-4b76-934a-3ea1426b0bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:13.966913 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4db218-0af2-4b76-934a-3ea1426b0bd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:13.967610 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4db218-0af2-4b76-934a-3ea1426b0bd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:13.969519 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4db218-0af2-4b76-934a-3ea1426b0bd2 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:13.974887 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 783/1578] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 13:46:13 2022] POST /identity/v3/projects => generated 385 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:46:13.975966 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2517ae7-c282-4110-98e0-e4e2be4a5bb1 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:14.000762 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 796/1579] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 13:46:13 2022] POST /identity/v3/projects => generated 401 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:46:14.000762 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4db218-0af2-4b76-934a-3ea1426b0bd2 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:14.019587 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:14.056563 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2517ae7-c282-4110-98e0-e4e2be4a5bb1 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:14.063344 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4db218-0af2-4b76-934a-3ea1426b0bd2 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:14.068822 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e2588e-5c04-43b2-ac07-10117ce71499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:14.069570 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e2588e-5c04-43b2-ac07-10117ce71499 admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:14.069908 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e2588e-5c04-43b2-ac07-10117ce71499 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:14.070208 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e2588e-5c04-43b2-ac07-10117ce71499 admin admin] PATH_INFO: `/v3/projects` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:14.070894 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e2588e-5c04-43b2-ac07-10117ce71499 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:14.072883 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e2588e-5c04-43b2-ac07-10117ce71499 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:14.102689 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 797/1580] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 13:46:13 2022] POST /identity/v3/projects => generated 391 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:46:14.106682 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e2588e-5c04-43b2-ac07-10117ce71499 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:14.119732 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:14.123468 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781d1337-60a8-4fe6-b2e4-49b510034d0d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c69161cd520b4b29bf50f3bf2f1c033b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:14.124108 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781d1337-60a8-4fe6-b2e4-49b510034d0d service nova] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:14.124397 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781d1337-60a8-4fe6-b2e4-49b510034d0d service nova] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:14.124676 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781d1337-60a8-4fe6-b2e4-49b510034d0d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:14.169174 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e2588e-5c04-43b2-ac07-10117ce71499 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:14.174083 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6730f6e3-a2b9-4e8c-ba47-45058262b9bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:14.174706 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6730f6e3-a2b9-4e8c-ba47-45058262b9bf admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:14.174988 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6730f6e3-a2b9-4e8c-ba47-45058262b9bf admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:14.175385 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6730f6e3-a2b9-4e8c-ba47-45058262b9bf admin admin] PATH_INFO: `/v3/projects` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:14.176055 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6730f6e3-a2b9-4e8c-ba47-45058262b9bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:14.178049 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6730f6e3-a2b9-4e8c-ba47-45058262b9bf admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:14.180882 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781d1337-60a8-4fe6-b2e4-49b510034d0d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:14.182498 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781d1337-60a8-4fe6-b2e4-49b510034d0d service nova] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:14.191997 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 784/1581] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:46:13 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:46:14.192636 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781d1337-60a8-4fe6-b2e4-49b510034d0d service nova] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:14.203421 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781d1337-60a8-4fe6-b2e4-49b510034d0d service nova] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:14.210565 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 798/1582] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 13:46:14 2022] POST /identity/v3/projects => generated 385 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:46:14.210565 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6730f6e3-a2b9-4e8c-ba47-45058262b9bf admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:14.212879 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125e3680-5188-4c0c-b2ba-6b443d640c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:14.213568 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125e3680-5188-4c0c-b2ba-6b443d640c8f admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:14.213916 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125e3680-5188-4c0c-b2ba-6b443d640c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:14.214125 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125e3680-5188-4c0c-b2ba-6b443d640c8f admin admin] PATH_INFO: `/v3/projects/e4a79afb56fa40598a8891c1fad1bd62/users/1747d40a285942eb9b4d5bc14136d88a/roles/759f389b221049c4b01faf3b984cd8d7` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:14.230410 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125e3680-5188-4c0c-b2ba-6b443d640c8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4a79afb56fa40598a8891c1fad1bd62, user_id=1747d40a285942eb9b4d5bc14136d88a, role_id=759f389b221049c4b01faf3b984cd8d7)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:14.233282 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125e3680-5188-4c0c-b2ba-6b443d640c8f admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:14.248247 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 785/1583] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 13:46:14 2022] PUT /identity/v3/projects/e4a79afb56fa40598a8891c1fad1bd62/users/1747d40a285942eb9b4d5bc14136d88a/roles/759f389b221049c4b01faf3b984cd8d7 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:46:14.249057 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125e3680-5188-4c0c-b2ba-6b443d640c8f admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:14.275544 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6730f6e3-a2b9-4e8c-ba47-45058262b9bf admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:14.301160 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:14.318001 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125e3680-5188-4c0c-b2ba-6b443d640c8f admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:14.322605 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdaa50e-168b-4cea-af9b-a0d6eb9cd2f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:14.323355 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdaa50e-168b-4cea-af9b-a0d6eb9cd2f5 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:14.323641 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdaa50e-168b-4cea-af9b-a0d6eb9cd2f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:14.323914 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdaa50e-168b-4cea-af9b-a0d6eb9cd2f5 admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:14.324572 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdaa50e-168b-4cea-af9b-a0d6eb9cd2f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:14.325610 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: WARNING py.warnings [None req-bbdaa50e-168b-4cea-af9b-a0d6eb9cd2f5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 13:46:14.325610 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: warnings.warn(msg) Jul 26 13:46:14.325610 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: Jul 26 13:46:14.326824 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdaa50e-168b-4cea-af9b-a0d6eb9cd2f5 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:14.339933 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a9c5a3-42ce-427d-a2c7-b932d9b243b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:14.341120 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a9c5a3-42ce-427d-a2c7-b932d9b243b0 admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:14.341729 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a9c5a3-42ce-427d-a2c7-b932d9b243b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:14.342068 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a9c5a3-42ce-427d-a2c7-b932d9b243b0 admin admin] PATH_INFO: `/v3/users` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:14.342934 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a9c5a3-42ce-427d-a2c7-b932d9b243b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:14.345120 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a9c5a3-42ce-427d-a2c7-b932d9b243b0 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:14.352712 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 786/1584] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 13:46:14 2022] POST /identity/v3/users => generated 289 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:46:14.352712 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdaa50e-168b-4cea-af9b-a0d6eb9cd2f5 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:14.364291 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdaa50e-168b-4cea-af9b-a0d6eb9cd2f5 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:14.369074 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61181789-35e0-41d0-89b0-7f39e2ccfb53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:14.369246 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 799/1585] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 13:46:14 2022] POST /identity/v3/users => generated 294 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:46:14.369573 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61181789-35e0-41d0-89b0-7f39e2ccfb53 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:14.369906 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61181789-35e0-41d0-89b0-7f39e2ccfb53 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:14.370181 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61181789-35e0-41d0-89b0-7f39e2ccfb53 admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:14.370972 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61181789-35e0-41d0-89b0-7f39e2ccfb53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:14.372643 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a9c5a3-42ce-427d-a2c7-b932d9b243b0 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:14.372895 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61181789-35e0-41d0-89b0-7f39e2ccfb53 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:14.385508 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a9c5a3-42ce-427d-a2c7-b932d9b243b0 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:14.391548 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe1919e-08e0-49da-bb98-b77cfddd8378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:14.392176 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe1919e-08e0-49da-bb98-b77cfddd8378 admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:14.393030 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe1919e-08e0-49da-bb98-b77cfddd8378 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:14.393299 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe1919e-08e0-49da-bb98-b77cfddd8378 admin admin] PATH_INFO: `/v3/users` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:14.394046 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe1919e-08e0-49da-bb98-b77cfddd8378 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:14.397641 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe1919e-08e0-49da-bb98-b77cfddd8378 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:14.399709 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 787/1586] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 13:46:14 2022] POST /identity/v3/users => generated 293 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:46:14.403444 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61181789-35e0-41d0-89b0-7f39e2ccfb53 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:14.414857 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61181789-35e0-41d0-89b0-7f39e2ccfb53 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:14.420502 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef789fa-45a2-4b3d-b8ae-39fa6a502fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:14.421220 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef789fa-45a2-4b3d-b8ae-39fa6a502fbc admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:14.421598 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef789fa-45a2-4b3d-b8ae-39fa6a502fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:14.421939 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef789fa-45a2-4b3d-b8ae-39fa6a502fbc admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:14.422666 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef789fa-45a2-4b3d-b8ae-39fa6a502fbc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:14.422976 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 800/1587] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 13:46:14 2022] POST /identity/v3/users => generated 301 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:46:14.424480 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe1919e-08e0-49da-bb98-b77cfddd8378 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:14.425404 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef789fa-45a2-4b3d-b8ae-39fa6a502fbc admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:14.436658 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe1919e-08e0-49da-bb98-b77cfddd8378 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:14.442543 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224b0df2-2bf6-4062-9034-4806004b75a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:14.443168 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224b0df2-2bf6-4062-9034-4806004b75a2 admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:14.443777 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224b0df2-2bf6-4062-9034-4806004b75a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:14.444057 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224b0df2-2bf6-4062-9034-4806004b75a2 admin admin] PATH_INFO: `/v3/users` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:14.444769 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224b0df2-2bf6-4062-9034-4806004b75a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:14.446746 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224b0df2-2bf6-4062-9034-4806004b75a2 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:14.449207 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 788/1588] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 13:46:14 2022] POST /identity/v3/users => generated 296 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:46:14.450498 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db217024-492e-4d94-8603-c3efc12d0d01 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:14.450788 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db217024-492e-4d94-8603-c3efc12d0d01 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:14.451068 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db217024-492e-4d94-8603-c3efc12d0d01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:14.463488 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:14.469375 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 801/1589] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 13:46:14 2022] POST /identity/v3/users => generated 293 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:46:14.470923 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-db217024-492e-4d94-8603-c3efc12d0d01 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:14.472290 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-db217024-492e-4d94-8603-c3efc12d0d01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:14.473839 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224b0df2-2bf6-4062-9034-4806004b75a2 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:14.480527 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 789/1590] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:46:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:46:14.481312 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db217024-492e-4d94-8603-c3efc12d0d01 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:14.486020 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224b0df2-2bf6-4062-9034-4806004b75a2 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:14.491274 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7733ce08-2936-4c1e-b1f6-543440bd5bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:14.491895 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7733ce08-2936-4c1e-b1f6-543440bd5bcf admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:14.492175 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7733ce08-2936-4c1e-b1f6-543440bd5bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:14.492447 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7733ce08-2936-4c1e-b1f6-543440bd5bcf admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:14.493142 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7733ce08-2936-4c1e-b1f6-543440bd5bcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:14.493976 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db217024-492e-4d94-8603-c3efc12d0d01 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:14.494776 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7733ce08-2936-4c1e-b1f6-543440bd5bcf admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:14.498762 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a30333-bdee-4d5b-9028-8637733885cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:14.499401 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a30333-bdee-4d5b-9028-8637733885cc admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:14.499683 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a30333-bdee-4d5b-9028-8637733885cc admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:14.499953 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a30333-bdee-4d5b-9028-8637733885cc admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:14.500641 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a30333-bdee-4d5b-9028-8637733885cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:14.502287 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a30333-bdee-4d5b-9028-8637733885cc admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:14.509940 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 802/1591] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:46:14 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:46:14.509940 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7733ce08-2936-4c1e-b1f6-543440bd5bcf admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:14.512505 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 790/1592] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:46:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:46:14.513263 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a30333-bdee-4d5b-9028-8637733885cc admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:14.520268 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7733ce08-2936-4c1e-b1f6-543440bd5bcf admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:14.525034 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd24fa1-11c8-4627-8c76-d0613f48db06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:14.525199 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a30333-bdee-4d5b-9028-8637733885cc admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:14.525685 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd24fa1-11c8-4627-8c76-d0613f48db06 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:14.525952 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd24fa1-11c8-4627-8c76-d0613f48db06 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:14.526227 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd24fa1-11c8-4627-8c76-d0613f48db06 admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:14.526909 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd24fa1-11c8-4627-8c76-d0613f48db06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:14.528458 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd24fa1-11c8-4627-8c76-d0613f48db06 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:14.529831 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ee0547-344d-42a4-ba1d-2f3f1c3579e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:14.530436 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ee0547-344d-42a4-ba1d-2f3f1c3579e1 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:14.530704 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ee0547-344d-42a4-ba1d-2f3f1c3579e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:14.530982 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ee0547-344d-42a4-ba1d-2f3f1c3579e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:14.531918 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ee0547-344d-42a4-ba1d-2f3f1c3579e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:14.534252 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ee0547-344d-42a4-ba1d-2f3f1c3579e1 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:14.539049 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 803/1593] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:46:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:46:14.540232 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7efd302-1894-48a7-b615-bf742c4c2587 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:14.540518 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7efd302-1894-48a7-b615-bf742c4c2587 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:14.540790 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7efd302-1894-48a7-b615-bf742c4c2587 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:14.542316 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 804/1594] 10.222.0.58 () {54 vars in 912 bytes} [Tue Jul 26 13:46:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:46:14.542977 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7efd302-1894-48a7-b615-bf742c4c2587 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:14.547052 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 791/1595] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:46:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:46:14.547052 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ee0547-344d-42a4-ba1d-2f3f1c3579e1 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:14.555098 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ee0547-344d-42a4-ba1d-2f3f1c3579e1 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:14.558447 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7efd302-1894-48a7-b615-bf742c4c2587 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:14.559586 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e230faf-d602-4721-9e42-ab0411fad0b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:14.560171 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e230faf-d602-4721-9e42-ab0411fad0b1 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:14.560454 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e230faf-d602-4721-9e42-ab0411fad0b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:14.560732 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e230faf-d602-4721-9e42-ab0411fad0b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:14.561409 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e230faf-d602-4721-9e42-ab0411fad0b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:14.563012 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e230faf-d602-4721-9e42-ab0411fad0b1 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:14.563404 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff0b973-ec05-4857-b29b-24f4dee13f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:14.564194 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff0b973-ec05-4857-b29b-24f4dee13f6c admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:14.564550 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff0b973-ec05-4857-b29b-24f4dee13f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:14.564895 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff0b973-ec05-4857-b29b-24f4dee13f6c admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:14.565695 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff0b973-ec05-4857-b29b-24f4dee13f6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:14.567259 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff0b973-ec05-4857-b29b-24f4dee13f6c admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:14.574824 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 792/1596] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:46:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:46:14.575642 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e230faf-d602-4721-9e42-ab0411fad0b1 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:14.577842 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 805/1597] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:46:14 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:46:14.578549 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff0b973-ec05-4857-b29b-24f4dee13f6c admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:14.588344 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff0b973-ec05-4857-b29b-24f4dee13f6c admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:14.592983 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dda95a-24f4-4f6c-b422-6836a3a43ca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:14.593626 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dda95a-24f4-4f6c-b422-6836a3a43ca0 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:14.593880 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dda95a-24f4-4f6c-b422-6836a3a43ca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:14.594190 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dda95a-24f4-4f6c-b422-6836a3a43ca0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:14.594834 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dda95a-24f4-4f6c-b422-6836a3a43ca0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:14.596944 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dda95a-24f4-4f6c-b422-6836a3a43ca0 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:14.606665 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 806/1598] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:46:14 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:46:14.607444 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dda95a-24f4-4f6c-b422-6836a3a43ca0 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:14.616908 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dda95a-24f4-4f6c-b422-6836a3a43ca0 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:14.621520 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f9b59b-d9d6-4085-888a-c9201071ec74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:14.622019 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f9b59b-d9d6-4085-888a-c9201071ec74 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:14.622300 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f9b59b-d9d6-4085-888a-c9201071ec74 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:14.622578 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f9b59b-d9d6-4085-888a-c9201071ec74 admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:14.623250 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f9b59b-d9d6-4085-888a-c9201071ec74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:14.624793 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f9b59b-d9d6-4085-888a-c9201071ec74 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:14.637341 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 807/1599] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:46:14 2022] GET /identity/v3/roles => generated 1420 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:46:14.637514 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e230faf-d602-4721-9e42-ab0411fad0b1 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:14.637617 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f9b59b-d9d6-4085-888a-c9201071ec74 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:14.639813 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233c0e65-4f9b-4c61-8e07-108aaf5fbb8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:14.640453 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233c0e65-4f9b-4c61-8e07-108aaf5fbb8c service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:14.640732 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233c0e65-4f9b-4c61-8e07-108aaf5fbb8c service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:14.641005 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233c0e65-4f9b-4c61-8e07-108aaf5fbb8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:14.647280 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f9b59b-d9d6-4085-888a-c9201071ec74 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:14.650739 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233c0e65-4f9b-4c61-8e07-108aaf5fbb8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:14.651761 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6455f576-4042-4e1e-b2c4-aa25ce1ee94c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:14.652152 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233c0e65-4f9b-4c61-8e07-108aaf5fbb8c service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:14.652334 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6455f576-4042-4e1e-b2c4-aa25ce1ee94c admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:14.652617 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6455f576-4042-4e1e-b2c4-aa25ce1ee94c admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:14.652890 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6455f576-4042-4e1e-b2c4-aa25ce1ee94c admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:14.653602 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6455f576-4042-4e1e-b2c4-aa25ce1ee94c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:14.655134 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6455f576-4042-4e1e-b2c4-aa25ce1ee94c admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:14.662438 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 793/1600] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:46:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:46:14.663687 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf00297-e2c9-4fec-9785-4a631ba5a9dd None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:14.663973 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf00297-e2c9-4fec-9785-4a631ba5a9dd None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:14.664249 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf00297-e2c9-4fec-9785-4a631ba5a9dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:14.666250 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 808/1601] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:46:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:46:14.667033 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6455f576-4042-4e1e-b2c4-aa25ce1ee94c admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:14.678596 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6455f576-4042-4e1e-b2c4-aa25ce1ee94c admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:14.683820 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3462c4e-95f4-4f6c-a1f7-1c8a7e9a38e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:14.684481 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3462c4e-95f4-4f6c-a1f7-1c8a7e9a38e2 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:14.685008 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3462c4e-95f4-4f6c-a1f7-1c8a7e9a38e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:14.685008 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3462c4e-95f4-4f6c-a1f7-1c8a7e9a38e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:14.685822 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3462c4e-95f4-4f6c-a1f7-1c8a7e9a38e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:14.686904 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-caf00297-e2c9-4fec-9785-4a631ba5a9dd None None] MFA Rules not processed for user `c69161cd520b4b29bf50f3bf2f1c033b`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:14.687490 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3462c4e-95f4-4f6c-a1f7-1c8a7e9a38e2 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:14.688204 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-caf00297-e2c9-4fec-9785-4a631ba5a9dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:14.705367 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 809/1602] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:46:14 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:46:14.705367 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3462c4e-95f4-4f6c-a1f7-1c8a7e9a38e2 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:14.713935 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3462c4e-95f4-4f6c-a1f7-1c8a7e9a38e2 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:14.719166 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bd4a3b-1b22-441c-9cb0-ea629d494f94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:14.719774 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bd4a3b-1b22-441c-9cb0-ea629d494f94 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:14.720181 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bd4a3b-1b22-441c-9cb0-ea629d494f94 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:14.721181 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bd4a3b-1b22-441c-9cb0-ea629d494f94 admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:14.721925 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bd4a3b-1b22-441c-9cb0-ea629d494f94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:14.723650 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bd4a3b-1b22-441c-9cb0-ea629d494f94 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:14.741103 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 810/1603] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:46:14 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:46:14.741103 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bd4a3b-1b22-441c-9cb0-ea629d494f94 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:14.753957 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bd4a3b-1b22-441c-9cb0-ea629d494f94 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:14.759650 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4b83e4-946a-4fb0-baf1-a871dfac32ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:14.760455 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4b83e4-946a-4fb0-baf1-a871dfac32ec admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:14.760892 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4b83e4-946a-4fb0-baf1-a871dfac32ec admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:14.761355 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4b83e4-946a-4fb0-baf1-a871dfac32ec admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:14.762397 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4b83e4-946a-4fb0-baf1-a871dfac32ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:14.762614 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:14.764298 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4b83e4-946a-4fb0-baf1-a871dfac32ec admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:14.779108 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 811/1604] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:46:14 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:46:14.779108 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4b83e4-946a-4fb0-baf1-a871dfac32ec admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:14.783657 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 794/1605] 10.222.0.58 () {58 vars in 1009 bytes} [Tue Jul 26 13:46:14 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:46:14.787737 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf00297-e2c9-4fec-9785-4a631ba5a9dd None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:14.792319 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4b83e4-946a-4fb0-baf1-a871dfac32ec admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:14.797132 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee9bdb0-e886-4792-afc7-d22352a5513b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:14.798333 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee9bdb0-e886-4792-afc7-d22352a5513b admin admin] REQUEST_METHOD: `PUT` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:14.799600 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee9bdb0-e886-4792-afc7-d22352a5513b admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:14.799600 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee9bdb0-e886-4792-afc7-d22352a5513b admin admin] PATH_INFO: `/v3/projects/629ae23d3b7b49cba61867a10e372a5c/users/1f5b2d80af624d45977460439d4eec2b/roles/759f389b221049c4b01faf3b984cd8d7` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:14.801164 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf00297-e2c9-4fec-9785-4a631ba5a9dd None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:14.807456 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeb4cfd-0b15-49e9-95de-45e6e5c35fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:14.808069 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeb4cfd-0b15-49e9-95de-45e6e5c35fac admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:14.808351 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeb4cfd-0b15-49e9-95de-45e6e5c35fac admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:14.808625 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeb4cfd-0b15-49e9-95de-45e6e5c35fac admin admin] PATH_INFO: `/v3/projects/560db1a629c4491abbc1742ddcec0747/users/cfbd6618ad654e3e812315455d21ccc7/roles/759f389b221049c4b01faf3b984cd8d7` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:14.818384 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee9bdb0-e886-4792-afc7-d22352a5513b admin admin] RBAC: Authorizing `identity:create_grant(project_id=629ae23d3b7b49cba61867a10e372a5c, user_id=1f5b2d80af624d45977460439d4eec2b, role_id=759f389b221049c4b01faf3b984cd8d7)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:14.821275 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee9bdb0-e886-4792-afc7-d22352a5513b admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:14.827069 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aeb4cfd-0b15-49e9-95de-45e6e5c35fac admin admin] RBAC: Authorizing `identity:create_grant(project_id=560db1a629c4491abbc1742ddcec0747, user_id=cfbd6618ad654e3e812315455d21ccc7, role_id=759f389b221049c4b01faf3b984cd8d7)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:14.830467 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aeb4cfd-0b15-49e9-95de-45e6e5c35fac admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:14.838924 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 812/1606] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 13:46:14 2022] PUT /identity/v3/projects/629ae23d3b7b49cba61867a10e372a5c/users/1f5b2d80af624d45977460439d4eec2b/roles/759f389b221049c4b01faf3b984cd8d7 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:46:14.839586 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee9bdb0-e886-4792-afc7-d22352a5513b admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:14.850941 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 795/1607] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 13:46:14 2022] PUT /identity/v3/projects/560db1a629c4491abbc1742ddcec0747/users/cfbd6618ad654e3e812315455d21ccc7/roles/759f389b221049c4b01faf3b984cd8d7 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:46:14.852033 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeb4cfd-0b15-49e9-95de-45e6e5c35fac admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:14.930065 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee9bdb0-e886-4792-afc7-d22352a5513b admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:14.936169 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dc0f81-07a2-4809-8283-971ffc37b142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:14.936328 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeb4cfd-0b15-49e9-95de-45e6e5c35fac admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:14.937574 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dc0f81-07a2-4809-8283-971ffc37b142 admin admin] REQUEST_METHOD: `PUT` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:14.938043 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dc0f81-07a2-4809-8283-971ffc37b142 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:14.938489 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dc0f81-07a2-4809-8283-971ffc37b142 admin admin] PATH_INFO: `/v3/projects/f833fde4e16046029aff7fb83f68b466/users/2b04d57398a44aaba82efa3015ac4983/roles/759f389b221049c4b01faf3b984cd8d7` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:14.943621 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2c363b-bbe9-4c91-94ba-33280fd31e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:14.944477 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2c363b-bbe9-4c91-94ba-33280fd31e6e admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:14.944829 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2c363b-bbe9-4c91-94ba-33280fd31e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:14.945173 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2c363b-bbe9-4c91-94ba-33280fd31e6e admin admin] PATH_INFO: `/v3/projects/48ca92b54ca442c09ecfcfa3af12469a/users/d4eec93dfb6f4a558888a2e9205ff003/roles/759f389b221049c4b01faf3b984cd8d7` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:14.956443 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dc0f81-07a2-4809-8283-971ffc37b142 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f833fde4e16046029aff7fb83f68b466, user_id=2b04d57398a44aaba82efa3015ac4983, role_id=759f389b221049c4b01faf3b984cd8d7)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:14.959424 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dc0f81-07a2-4809-8283-971ffc37b142 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:14.961945 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2c363b-bbe9-4c91-94ba-33280fd31e6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=48ca92b54ca442c09ecfcfa3af12469a, user_id=d4eec93dfb6f4a558888a2e9205ff003, role_id=759f389b221049c4b01faf3b984cd8d7)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:14.964977 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2c363b-bbe9-4c91-94ba-33280fd31e6e admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:14.974449 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 813/1608] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 13:46:14 2022] PUT /identity/v3/projects/f833fde4e16046029aff7fb83f68b466/users/2b04d57398a44aaba82efa3015ac4983/roles/759f389b221049c4b01faf3b984cd8d7 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:46:14.985975 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dc0f81-07a2-4809-8283-971ffc37b142 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:14.988620 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 796/1609] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 13:46:14 2022] PUT /identity/v3/projects/48ca92b54ca442c09ecfcfa3af12469a/users/d4eec93dfb6f4a558888a2e9205ff003/roles/759f389b221049c4b01faf3b984cd8d7 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:46:14.989519 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2c363b-bbe9-4c91-94ba-33280fd31e6e admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:15.015687 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:15.077234 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dc0f81-07a2-4809-8283-971ffc37b142 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:15.090164 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2c363b-bbe9-4c91-94ba-33280fd31e6e admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:15.095870 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92270b86-89f3-44fc-aa23-e0f9eb67a4df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:15.096562 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92270b86-89f3-44fc-aa23-e0f9eb67a4df admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:15.096844 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92270b86-89f3-44fc-aa23-e0f9eb67a4df admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:15.097118 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92270b86-89f3-44fc-aa23-e0f9eb67a4df admin admin] PATH_INFO: `/v3/projects/78ae845c827f4448ad340221d5816fd1/users/0accc3551cd348e2a946bde38e4db40f/roles/759f389b221049c4b01faf3b984cd8d7` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:15.114836 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92270b86-89f3-44fc-aa23-e0f9eb67a4df admin admin] RBAC: Authorizing `identity:create_grant(project_id=78ae845c827f4448ad340221d5816fd1, user_id=0accc3551cd348e2a946bde38e4db40f, role_id=759f389b221049c4b01faf3b984cd8d7)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:15.117632 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92270b86-89f3-44fc-aa23-e0f9eb67a4df admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:15.134741 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 797/1610] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 13:46:14 2022] PUT /identity/v3/projects/78ae845c827f4448ad340221d5816fd1/users/0accc3551cd348e2a946bde38e4db40f/roles/759f389b221049c4b01faf3b984cd8d7 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:46:15.135783 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92270b86-89f3-44fc-aa23-e0f9eb67a4df admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:15.146731 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e189ee8-4332-4dc5-8a92-065ee580f38b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:15.147796 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e189ee8-4332-4dc5-8a92-065ee580f38b admin admin] REQUEST_METHOD: `PUT` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:15.148296 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e189ee8-4332-4dc5-8a92-065ee580f38b admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:15.148680 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e189ee8-4332-4dc5-8a92-065ee580f38b admin admin] PATH_INFO: `/v3/projects/f4814f42888d4554a082b2bdb2036ac8/users/56db6603f359477885b5151dec71c096/roles/759f389b221049c4b01faf3b984cd8d7` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:15.165225 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e189ee8-4332-4dc5-8a92-065ee580f38b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4814f42888d4554a082b2bdb2036ac8, user_id=56db6603f359477885b5151dec71c096, role_id=759f389b221049c4b01faf3b984cd8d7)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:15.168044 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e189ee8-4332-4dc5-8a92-065ee580f38b admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:15.188082 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:15.188914 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 814/1611] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 13:46:14 2022] PUT /identity/v3/projects/f4814f42888d4554a082b2bdb2036ac8/users/56db6603f359477885b5151dec71c096/roles/759f389b221049c4b01faf3b984cd8d7 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:46:15.190419 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af36ab1a-28ae-4c6d-aebf-e5463d607e22 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:15.190813 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af36ab1a-28ae-4c6d-aebf-e5463d607e22 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:15.191190 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af36ab1a-28ae-4c6d-aebf-e5463d607e22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:15.206272 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92270b86-89f3-44fc-aa23-e0f9eb67a4df admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:15.211250 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-af36ab1a-28ae-4c6d-aebf-e5463d607e22 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:15.212537 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-af36ab1a-28ae-4c6d-aebf-e5463d607e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:15.261905 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca236a4-06d7-47a6-8347-586dc97ec0df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:15.262537 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca236a4-06d7-47a6-8347-586dc97ec0df service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:15.262866 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca236a4-06d7-47a6-8347-586dc97ec0df service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:15.263274 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca236a4-06d7-47a6-8347-586dc97ec0df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:15.285163 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 815/1612] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:46:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:46:15.286661 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b5b07a-e153-41c0-9449-954e85b221f0 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:15.287011 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b5b07a-e153-41c0-9449-954e85b221f0 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:15.287351 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b5b07a-e153-41c0-9449-954e85b221f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:15.323504 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-19b5b07a-e153-41c0-9449-954e85b221f0 None None] MFA Rules not processed for user `cfbd6618ad654e3e812315455d21ccc7`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:15.325016 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-19b5b07a-e153-41c0-9449-954e85b221f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:15.337003 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca236a4-06d7-47a6-8347-586dc97ec0df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:15.338622 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca236a4-06d7-47a6-8347-586dc97ec0df service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:15.351010 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 798/1613] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:46:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:46:15.354158 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e64135-e3ec-4979-964c-7852c019872f None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:15.354439 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e64135-e3ec-4979-964c-7852c019872f None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:15.354716 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e64135-e3ec-4979-964c-7852c019872f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:15.392587 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-e8e64135-e3ec-4979-964c-7852c019872f None None] MFA Rules not processed for user `2b04d57398a44aaba82efa3015ac4983`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:15.395907 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-e8e64135-e3ec-4979-964c-7852c019872f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:15.400146 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:15.436303 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 816/1614] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:46:15 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:46:15.439127 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d885bc4-af07-4636-9a4d-0ac45bcfe49c None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:15.439643 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d885bc4-af07-4636-9a4d-0ac45bcfe49c None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:15.440813 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d885bc4-af07-4636-9a4d-0ac45bcfe49c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:15.479079 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 799/1615] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:46:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:46:15.479410 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-5d885bc4-af07-4636-9a4d-0ac45bcfe49c None None] MFA Rules not processed for user `d4eec93dfb6f4a558888a2e9205ff003`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:15.480487 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964e5f2b-fa08-4d5d-ae0b-a82984da2ef4 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:15.480865 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-5d885bc4-af07-4636-9a4d-0ac45bcfe49c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:15.480995 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964e5f2b-fa08-4d5d-ae0b-a82984da2ef4 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:15.481315 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964e5f2b-fa08-4d5d-ae0b-a82984da2ef4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:15.505370 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-964e5f2b-fa08-4d5d-ae0b-a82984da2ef4 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:15.506682 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-964e5f2b-fa08-4d5d-ae0b-a82984da2ef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:15.520123 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 800/1616] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:46:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:46:15.520123 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3babbb6-34c5-42c0-a1ad-d3cf50da4357 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:15.520123 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3babbb6-34c5-42c0-a1ad-d3cf50da4357 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:15.520123 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3babbb6-34c5-42c0-a1ad-d3cf50da4357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:15.563860 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-e3babbb6-34c5-42c0-a1ad-d3cf50da4357 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:15.569898 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-e3babbb6-34c5-42c0-a1ad-d3cf50da4357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:15.591026 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 801/1617] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:46:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:46:15.596004 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3babbb6-34c5-42c0-a1ad-d3cf50da4357 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:15.620335 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 817/1618] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:46:15 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:46:15.621287 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d885bc4-af07-4636-9a4d-0ac45bcfe49c None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:15.625007 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:15.637692 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d885bc4-af07-4636-9a4d-0ac45bcfe49c None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:15.644599 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e480210-3ef7-4c2d-9e33-1a137acc8324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:15.645588 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e480210-3ef7-4c2d-9e33-1a137acc8324 admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:15.645982 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e480210-3ef7-4c2d-9e33-1a137acc8324 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:15.646342 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e480210-3ef7-4c2d-9e33-1a137acc8324 admin admin] PATH_INFO: `/v3/projects` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:15.647264 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e480210-3ef7-4c2d-9e33-1a137acc8324 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:15.649951 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e480210-3ef7-4c2d-9e33-1a137acc8324 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:15.698083 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 818/1619] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 13:46:15 2022] POST /identity/v3/projects => generated 387 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:46:15.699810 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6740d268-c7a6-4ce2-89a7-60c8a3baf547 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:15.700098 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6740d268-c7a6-4ce2-89a7-60c8a3baf547 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:15.700373 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6740d268-c7a6-4ce2-89a7-60c8a3baf547 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:15.705294 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 819/1620] 10.222.0.58 () {54 vars in 962 bytes} [Tue Jul 26 13:46:15 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:46:15.712894 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6740d268-c7a6-4ce2-89a7-60c8a3baf547 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:15.717174 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-6740d268-c7a6-4ce2-89a7-60c8a3baf547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:15.738857 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3babbb6-34c5-42c0-a1ad-d3cf50da4357 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:15.813457 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69548ed-a679-4020-ab2e-f0f082c46679 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:15.814483 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69548ed-a679-4020-ab2e-f0f082c46679 service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:15.815025 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69548ed-a679-4020-ab2e-f0f082c46679 service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:15.815409 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69548ed-a679-4020-ab2e-f0f082c46679 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:15.872609 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6740d268-c7a6-4ce2-89a7-60c8a3baf547 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:15.882805 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41398e73-dd15-4f38-ae7b-92566cd86af2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:15.883906 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41398e73-dd15-4f38-ae7b-92566cd86af2 service neutron] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:15.884193 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41398e73-dd15-4f38-ae7b-92566cd86af2 service neutron] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:15.884474 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41398e73-dd15-4f38-ae7b-92566cd86af2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:15.899357 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69548ed-a679-4020-ab2e-f0f082c46679 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:15.902616 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41398e73-dd15-4f38-ae7b-92566cd86af2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:15.904354 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41398e73-dd15-4f38-ae7b-92566cd86af2 service neutron] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:15.905261 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69548ed-a679-4020-ab2e-f0f082c46679 service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:15.927146 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 820/1621] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:46:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:46:15.928492 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41398e73-dd15-4f38-ae7b-92566cd86af2 service neutron] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:15.934338 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 802/1622] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:46:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:46:15.935081 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69548ed-a679-4020-ab2e-f0f082c46679 service neutron] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:15.949955 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41398e73-dd15-4f38-ae7b-92566cd86af2 service neutron] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:15.953671 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69548ed-a679-4020-ab2e-f0f082c46679 service neutron] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:15.960317 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d6610a-41aa-4c13-8e52-5830f1e6ece9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:15.961205 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d6610a-41aa-4c13-8e52-5830f1e6ece9 service neutron] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:15.961548 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d6610a-41aa-4c13-8e52-5830f1e6ece9 service neutron] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:15.961792 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d6610a-41aa-4c13-8e52-5830f1e6ece9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:15.965282 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e6450f-04bf-46f1-bdab-d83e223f0c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:15.967273 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e6450f-04bf-46f1-bdab-d83e223f0c2d admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:15.967569 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e6450f-04bf-46f1-bdab-d83e223f0c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:15.967868 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e6450f-04bf-46f1-bdab-d83e223f0c2d admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:15.968525 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e6450f-04bf-46f1-bdab-d83e223f0c2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:15.975069 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e6450f-04bf-46f1-bdab-d83e223f0c2d admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:15.990338 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d6610a-41aa-4c13-8e52-5830f1e6ece9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:15.992279 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d6610a-41aa-4c13-8e52-5830f1e6ece9 service neutron] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:16.016643 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 821/1623] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:46:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:46:16.018100 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d6610a-41aa-4c13-8e52-5830f1e6ece9 service neutron] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:16.044307 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 803/1624] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 13:46:15 2022] POST /identity/v3/projects => generated 405 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:46:16.046793 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1695b946-8f77-42cb-ab27-22e1dfec83b4 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:16.049459 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1695b946-8f77-42cb-ab27-22e1dfec83b4 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:16.049777 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1695b946-8f77-42cb-ab27-22e1dfec83b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:16.071093 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d6610a-41aa-4c13-8e52-5830f1e6ece9 service neutron] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:16.106769 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-1695b946-8f77-42cb-ab27-22e1dfec83b4 None None] MFA Rules not processed for user `c69161cd520b4b29bf50f3bf2f1c033b`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:16.109709 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-1695b946-8f77-42cb-ab27-22e1dfec83b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:16.152852 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:16.187624 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2d68d7-1950-4c11-b74f-d48c226d7cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:16.189868 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2d68d7-1950-4c11-b74f-d48c226d7cd5 admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:16.190452 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2d68d7-1950-4c11-b74f-d48c226d7cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:16.191024 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2d68d7-1950-4c11-b74f-d48c226d7cd5 admin admin] PATH_INFO: `/v3/users` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:16.191868 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2d68d7-1950-4c11-b74f-d48c226d7cd5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:16.194864 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2d68d7-1950-4c11-b74f-d48c226d7cd5 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:16.241787 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 822/1625] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 13:46:16 2022] POST /identity/v3/users => generated 300 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:46:16.244091 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2d68d7-1950-4c11-b74f-d48c226d7cd5 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:16.268406 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2d68d7-1950-4c11-b74f-d48c226d7cd5 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:16.279820 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cb1141-f1d7-4515-b9bd-ea5d57afd6ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:16.282196 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cb1141-f1d7-4515-b9bd-ea5d57afd6ca admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:16.287452 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cb1141-f1d7-4515-b9bd-ea5d57afd6ca admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:16.287938 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cb1141-f1d7-4515-b9bd-ea5d57afd6ca admin admin] PATH_INFO: `/v3/users` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:16.288808 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cb1141-f1d7-4515-b9bd-ea5d57afd6ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:16.289432 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:16.293670 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cb1141-f1d7-4515-b9bd-ea5d57afd6ca admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:16.335148 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 804/1626] 10.222.0.58 () {58 vars in 1059 bytes} [Tue Jul 26 13:46:16 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:46:16.341759 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1695b946-8f77-42cb-ab27-22e1dfec83b4 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:16.364523 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1695b946-8f77-42cb-ab27-22e1dfec83b4 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:16.368918 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 823/1627] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 13:46:16 2022] POST /identity/v3/users => generated 309 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:46:16.368918 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cb1141-f1d7-4515-b9bd-ea5d57afd6ca admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:16.378628 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0d3753-1857-4eca-b6f0-1f4943521be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:16.380833 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0d3753-1857-4eca-b6f0-1f4943521be1 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:16.381993 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0d3753-1857-4eca-b6f0-1f4943521be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:16.383303 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0d3753-1857-4eca-b6f0-1f4943521be1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:16.385222 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0d3753-1857-4eca-b6f0-1f4943521be1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:16.389826 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0d3753-1857-4eca-b6f0-1f4943521be1 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:16.401710 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cb1141-f1d7-4515-b9bd-ea5d57afd6ca admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:16.415879 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85810689-d352-465c-a7ef-b40e92617475 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c69161cd520b4b29bf50f3bf2f1c033b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:16.417805 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85810689-d352-465c-a7ef-b40e92617475 service nova] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:16.418091 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85810689-d352-465c-a7ef-b40e92617475 service nova] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:16.418368 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85810689-d352-465c-a7ef-b40e92617475 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:16.422719 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 805/1628] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:46:16 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:46:16.422719 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0d3753-1857-4eca-b6f0-1f4943521be1 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:16.446920 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0d3753-1857-4eca-b6f0-1f4943521be1 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:16.463083 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73c2ef9-12c4-4cb2-ac8f-2831ccc083cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:16.463789 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73c2ef9-12c4-4cb2-ac8f-2831ccc083cd admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:16.464032 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73c2ef9-12c4-4cb2-ac8f-2831ccc083cd admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:16.467867 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73c2ef9-12c4-4cb2-ac8f-2831ccc083cd admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:16.472666 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73c2ef9-12c4-4cb2-ac8f-2831ccc083cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:16.474194 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73c2ef9-12c4-4cb2-ac8f-2831ccc083cd admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:16.499643 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 806/1629] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:46:16 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:46:16.504390 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73c2ef9-12c4-4cb2-ac8f-2831ccc083cd admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:16.526458 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73c2ef9-12c4-4cb2-ac8f-2831ccc083cd admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:16.543223 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b890df4-b6fa-4b8d-a31e-39505c8c9a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:16.543905 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b890df4-b6fa-4b8d-a31e-39505c8c9a06 admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:16.544186 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b890df4-b6fa-4b8d-a31e-39505c8c9a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:16.544564 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b890df4-b6fa-4b8d-a31e-39505c8c9a06 admin admin] PATH_INFO: `/v3/projects/d194af8db3114da7b5f2e5741f428461/users/fa56661a48f046e3918444fc71111689/roles/6e0201a9a6484f7d8b112b1eafff70bd` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:16.571105 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b890df4-b6fa-4b8d-a31e-39505c8c9a06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d194af8db3114da7b5f2e5741f428461, user_id=fa56661a48f046e3918444fc71111689, role_id=6e0201a9a6484f7d8b112b1eafff70bd)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:16.573981 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b890df4-b6fa-4b8d-a31e-39505c8c9a06 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:16.581688 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85810689-d352-465c-a7ef-b40e92617475 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:16.583627 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85810689-d352-465c-a7ef-b40e92617475 service nova] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:16.598586 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 807/1630] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 13:46:16 2022] PUT /identity/v3/projects/d194af8db3114da7b5f2e5741f428461/users/fa56661a48f046e3918444fc71111689/roles/6e0201a9a6484f7d8b112b1eafff70bd => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:46:16.600108 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b890df4-b6fa-4b8d-a31e-39505c8c9a06 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:16.610591 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 824/1631] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:46:16 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:46:16.612315 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85810689-d352-465c-a7ef-b40e92617475 service nova] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:16.637494 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85810689-d352-465c-a7ef-b40e92617475 service nova] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:16.823614 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b890df4-b6fa-4b8d-a31e-39505c8c9a06 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:16.823614 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af014edd-4325-440b-ab8d-e8cb8106a1d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:16.823614 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af014edd-4325-440b-ab8d-e8cb8106a1d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:16.823614 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af014edd-4325-440b-ab8d-e8cb8106a1d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:16.825290 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af014edd-4325-440b-ab8d-e8cb8106a1d2 admin admin] PATH_INFO: `/v3/projects/54f5e370c14a44f882b2f8e2dbbb781e/users/149d925dcd354812b3e0dbfdfdb87647/roles/6e0201a9a6484f7d8b112b1eafff70bd` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:16.826246 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6f7f26-ca43-4e2a-8259-aa09910fcf72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:16.826246 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f7f26-ca43-4e2a-8259-aa09910fcf72 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:16.826246 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f7f26-ca43-4e2a-8259-aa09910fcf72 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:16.826246 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f7f26-ca43-4e2a-8259-aa09910fcf72 admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:16.827021 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6f7f26-ca43-4e2a-8259-aa09910fcf72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:16.827021 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6f7f26-ca43-4e2a-8259-aa09910fcf72 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:16.828196 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af014edd-4325-440b-ab8d-e8cb8106a1d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54f5e370c14a44f882b2f8e2dbbb781e, user_id=149d925dcd354812b3e0dbfdfdb87647, role_id=6e0201a9a6484f7d8b112b1eafff70bd)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:16.828196 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af014edd-4325-440b-ab8d-e8cb8106a1d2 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:16.828649 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 825/1632] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:46:16 2022] GET /identity/v3/roles => generated 1420 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:46:16.830711 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6f7f26-ca43-4e2a-8259-aa09910fcf72 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:16.854899 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6f7f26-ca43-4e2a-8259-aa09910fcf72 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:16.856489 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 808/1633] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 13:46:16 2022] PUT /identity/v3/projects/54f5e370c14a44f882b2f8e2dbbb781e/users/149d925dcd354812b3e0dbfdfdb87647/roles/6e0201a9a6484f7d8b112b1eafff70bd => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:46:16.862114 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af014edd-4325-440b-ab8d-e8cb8106a1d2 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:16.918369 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:16.960930 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1572a39b-4697-48fb-a321-eed7abd2a36c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:16.961699 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1572a39b-4697-48fb-a321-eed7abd2a36c admin admin] REQUEST_METHOD: `PUT` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:16.963154 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1572a39b-4697-48fb-a321-eed7abd2a36c admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:16.963714 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1572a39b-4697-48fb-a321-eed7abd2a36c admin admin] PATH_INFO: `/v3/projects/d194af8db3114da7b5f2e5741f428461/users/fa56661a48f046e3918444fc71111689/roles/6e0201a9a6484f7d8b112b1eafff70bd` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:16.983050 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1572a39b-4697-48fb-a321-eed7abd2a36c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d194af8db3114da7b5f2e5741f428461, user_id=fa56661a48f046e3918444fc71111689, role_id=6e0201a9a6484f7d8b112b1eafff70bd)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:16.988637 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1572a39b-4697-48fb-a321-eed7abd2a36c admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:17.019155 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 826/1634] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 13:46:16 2022] PUT /identity/v3/projects/d194af8db3114da7b5f2e5741f428461/users/fa56661a48f046e3918444fc71111689/roles/6e0201a9a6484f7d8b112b1eafff70bd => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:46:17.025163 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af014edd-4325-440b-ab8d-e8cb8106a1d2 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:17.030302 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c34029a-971e-4346-bc6d-c819e9c0d908 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:17.030754 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c34029a-971e-4346-bc6d-c819e9c0d908 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:17.035137 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c34029a-971e-4346-bc6d-c819e9c0d908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:17.096632 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-7c34029a-971e-4346-bc6d-c819e9c0d908 None None] MFA Rules not processed for user `fa56661a48f046e3918444fc71111689`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:17.099548 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-7c34029a-971e-4346-bc6d-c819e9c0d908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:17.108296 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:17.146458 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f258f345-760e-4276-b081-f71f5fe4be40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:17.147147 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f258f345-760e-4276-b081-f71f5fe4be40 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:17.147440 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f258f345-760e-4276-b081-f71f5fe4be40 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:17.147713 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f258f345-760e-4276-b081-f71f5fe4be40 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:17.148706 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f258f345-760e-4276-b081-f71f5fe4be40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:17.150926 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f258f345-760e-4276-b081-f71f5fe4be40 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:17.164958 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 809/1635] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:46:16 2022] GET /identity/v3/roles => generated 1420 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 13:46:17.167618 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c05cb9-b64a-4c0e-8555-c2d32b0dac61 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:17.168193 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c05cb9-b64a-4c0e-8555-c2d32b0dac61 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:17.168902 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c05cb9-b64a-4c0e-8555-c2d32b0dac61 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:17.178021 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 810/1636] 10.222.0.58 () {54 vars in 912 bytes} [Tue Jul 26 13:46:17 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:46:17.179423 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c05cb9-b64a-4c0e-8555-c2d32b0dac61 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:17.197172 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c05cb9-b64a-4c0e-8555-c2d32b0dac61 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:17.203124 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937b0b4e-016d-4311-8bdb-f2f2fd2aa0da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:17.203970 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937b0b4e-016d-4311-8bdb-f2f2fd2aa0da admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:17.204259 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937b0b4e-016d-4311-8bdb-f2f2fd2aa0da admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:17.204554 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937b0b4e-016d-4311-8bdb-f2f2fd2aa0da admin admin] PATH_INFO: `/v3/projects/54f5e370c14a44f882b2f8e2dbbb781e/users/149d925dcd354812b3e0dbfdfdb87647/roles/6e0201a9a6484f7d8b112b1eafff70bd` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:17.207534 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937b0b4e-016d-4311-8bdb-f2f2fd2aa0da admin admin] RBAC: Authorizing `identity:create_grant(project_id=54f5e370c14a44f882b2f8e2dbbb781e, user_id=149d925dcd354812b3e0dbfdfdb87647, role_id=6e0201a9a6484f7d8b112b1eafff70bd)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:17.210667 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937b0b4e-016d-4311-8bdb-f2f2fd2aa0da admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:17.223418 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 827/1637] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:46:17 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:46:17.225177 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a954ffe-719c-4c5e-b9a5-d5c3918c918b None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:17.228247 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a954ffe-719c-4c5e-b9a5-d5c3918c918b None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:17.228674 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a954ffe-719c-4c5e-b9a5-d5c3918c918b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:17.247456 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 811/1638] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 13:46:17 2022] PUT /identity/v3/projects/54f5e370c14a44f882b2f8e2dbbb781e/users/149d925dcd354812b3e0dbfdfdb87647/roles/6e0201a9a6484f7d8b112b1eafff70bd => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:46:17.255444 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2f3078-48b5-4676-9c31-e735c3ee5769 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:17.255984 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2f3078-48b5-4676-9c31-e735c3ee5769 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:17.256328 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2f3078-48b5-4676-9c31-e735c3ee5769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:17.290739 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-db2f3078-48b5-4676-9c31-e735c3ee5769 None None] MFA Rules not processed for user `149d925dcd354812b3e0dbfdfdb87647`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:17.291955 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-db2f3078-48b5-4676-9c31-e735c3ee5769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:17.297592 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-6a954ffe-719c-4c5e-b9a5-d5c3918c918b None None] MFA Rules not processed for user `c69161cd520b4b29bf50f3bf2f1c033b`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:17.304358 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-6a954ffe-719c-4c5e-b9a5-d5c3918c918b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:17.396859 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 812/1639] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:46:17 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:46:17.397835 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2f3078-48b5-4676-9c31-e735c3ee5769 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:17.487517 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 828/1640] 10.222.0.58 () {58 vars in 1009 bytes} [Tue Jul 26 13:46:17 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:46:17.489754 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a954ffe-719c-4c5e-b9a5-d5c3918c918b None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:17.523238 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2f3078-48b5-4676-9c31-e735c3ee5769 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:17.549011 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d679a58-8fce-447b-b8b1-e5c02472cf3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:17.550121 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d679a58-8fce-447b-b8b1-e5c02472cf3c service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:17.551316 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d679a58-8fce-447b-b8b1-e5c02472cf3c service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:17.551712 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d679a58-8fce-447b-b8b1-e5c02472cf3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:17.666417 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a954ffe-719c-4c5e-b9a5-d5c3918c918b None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:17.674617 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aba10b-ad62-4c04-9b89-5f9ee847e67a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:17.675238 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aba10b-ad62-4c04-9b89-5f9ee847e67a service cinder] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:17.675690 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aba10b-ad62-4c04-9b89-5f9ee847e67a service cinder] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:17.675993 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aba10b-ad62-4c04-9b89-5f9ee847e67a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:17.694178 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:17.696096 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7aba10b-ad62-4c04-9b89-5f9ee847e67a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:17.698108 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7aba10b-ad62-4c04-9b89-5f9ee847e67a service cinder] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:17.705948 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:17.719915 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d679a58-8fce-447b-b8b1-e5c02472cf3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:17.722132 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d679a58-8fce-447b-b8b1-e5c02472cf3c service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:17.727139 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 829/1641] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:46:17 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:46:17.745951 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 813/1642] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:46:17 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:46:17.821569 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aba10b-ad62-4c04-9b89-5f9ee847e67a service cinder] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:17.868048 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d679a58-8fce-447b-b8b1-e5c02472cf3c service cinder] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:17.878760 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d679a58-8fce-447b-b8b1-e5c02472cf3c service cinder] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:17.899227 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cc62f6-53fa-48f8-a033-99a99302d110 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:17.899925 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cc62f6-53fa-48f8-a033-99a99302d110 service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:17.900245 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cc62f6-53fa-48f8-a033-99a99302d110 service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:17.900524 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cc62f6-53fa-48f8-a033-99a99302d110 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:17.939306 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cc62f6-53fa-48f8-a033-99a99302d110 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:17.940684 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cc62f6-53fa-48f8-a033-99a99302d110 service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:17.994374 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 814/1643] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:46:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 13:46:18.034291 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aba10b-ad62-4c04-9b89-5f9ee847e67a service cinder] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:18.052714 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f76aa6-bdd3-4650-a685-58101bd8230f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:18.053994 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f76aa6-bdd3-4650-a685-58101bd8230f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:18.055726 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f76aa6-bdd3-4650-a685-58101bd8230f admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:18.056138 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f76aa6-bdd3-4650-a685-58101bd8230f admin admin] PATH_INFO: `/v3/users/cfbd6618ad654e3e812315455d21ccc7` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:18.058699 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f76aa6-bdd3-4650-a685-58101bd8230f admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfbd6618ad654e3e812315455d21ccc7)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:18.064045 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: WARNING py.warnings [None req-59f76aa6-bdd3-4650-a685-58101bd8230f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 13:46:18.064045 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: warnings.warn(msg) Jul 26 13:46:18.064045 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: Jul 26 13:46:18.067708 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f76aa6-bdd3-4650-a685-58101bd8230f admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:18.215723 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-59f76aa6-bdd3-4650-a685-58101bd8230f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfbd6618ad654e3e812315455d21ccc7'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:18.239861 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-59f76aa6-bdd3-4650-a685-58101bd8230f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfbd6618ad654e3e812315455d21ccc7'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:18.269045 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-59f76aa6-bdd3-4650-a685-58101bd8230f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfbd6618ad654e3e812315455d21ccc7'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:18.270831 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-59f76aa6-bdd3-4650-a685-58101bd8230f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfbd6618ad654e3e812315455d21ccc7'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:18.271478 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-59f76aa6-bdd3-4650-a685-58101bd8230f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfbd6618ad654e3e812315455d21ccc7'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:18.346983 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cc62f6-53fa-48f8-a033-99a99302d110 service cinder] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:18.349443 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 830/1644] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 13:46:17 2022] DELETE /identity/v3/users/cfbd6618ad654e3e812315455d21ccc7 => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:46:18.354358 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0867a4-604a-44a4-b06d-bc1a3a55eee0 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:18.354643 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0867a4-604a-44a4-b06d-bc1a3a55eee0 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:18.354918 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0867a4-604a-44a4-b06d-bc1a3a55eee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:18.365535 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cc62f6-53fa-48f8-a033-99a99302d110 service cinder] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:18.411027 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-db0867a4-604a-44a4-b06d-bc1a3a55eee0 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:18.417132 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-db0867a4-604a-44a4-b06d-bc1a3a55eee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:18.482904 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dbe58d-44eb-465c-88d5-7be37ee31c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:18.483622 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dbe58d-44eb-465c-88d5-7be37ee31c35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:18.483853 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dbe58d-44eb-465c-88d5-7be37ee31c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:18.484130 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dbe58d-44eb-465c-88d5-7be37ee31c35 admin admin] PATH_INFO: `/v3/users/d4eec93dfb6f4a558888a2e9205ff003` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:18.485359 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dbe58d-44eb-465c-88d5-7be37ee31c35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4eec93dfb6f4a558888a2e9205ff003)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:18.487506 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dbe58d-44eb-465c-88d5-7be37ee31c35 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:18.564144 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-e5dbe58d-44eb-465c-88d5-7be37ee31c35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4eec93dfb6f4a558888a2e9205ff003'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:18.570798 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-e5dbe58d-44eb-465c-88d5-7be37ee31c35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4eec93dfb6f4a558888a2e9205ff003'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:18.588731 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-e5dbe58d-44eb-465c-88d5-7be37ee31c35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4eec93dfb6f4a558888a2e9205ff003'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:18.589906 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-e5dbe58d-44eb-465c-88d5-7be37ee31c35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4eec93dfb6f4a558888a2e9205ff003'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:18.602425 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 831/1645] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:46:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:46:18.613625 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-e5dbe58d-44eb-465c-88d5-7be37ee31c35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4eec93dfb6f4a558888a2e9205ff003'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:18.651077 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0867a4-604a-44a4-b06d-bc1a3a55eee0 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:18.652738 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-db0867a4-604a-44a4-b06d-bc1a3a55eee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:18.663602 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 815/1646] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 13:46:18 2022] DELETE /identity/v3/users/d4eec93dfb6f4a558888a2e9205ff003 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 13:46:18.668998 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bfde27-cafa-411a-8cac-8d58f10e5056 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:18.670260 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bfde27-cafa-411a-8cac-8d58f10e5056 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:18.671576 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bfde27-cafa-411a-8cac-8d58f10e5056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:18.702361 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:18.724517 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-51bfde27-cafa-411a-8cac-8d58f10e5056 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:18.728234 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-51bfde27-cafa-411a-8cac-8d58f10e5056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:18.801293 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0867a4-604a-44a4-b06d-bc1a3a55eee0 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:18.854020 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:18.924680 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 816/1647] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:46:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:46:18.974809 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3111a96-7d89-472e-abf3-31e0a4050b5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:18.975713 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3111a96-7d89-472e-abf3-31e0a4050b5f service neutron] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:18.976259 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3111a96-7d89-472e-abf3-31e0a4050b5f service neutron] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:18.977130 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3111a96-7d89-472e-abf3-31e0a4050b5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:18.992918 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3111a96-7d89-472e-abf3-31e0a4050b5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:18.994960 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3111a96-7d89-472e-abf3-31e0a4050b5f service neutron] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:19.019536 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 832/1648] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:46:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 13:46:19.085917 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bfde27-cafa-411a-8cac-8d58f10e5056 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:19.087495 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-51bfde27-cafa-411a-8cac-8d58f10e5056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:19.135036 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bfde27-cafa-411a-8cac-8d58f10e5056 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:19.140531 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2231ce7c-c5e9-40b5-a701-7d447c94a42b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:19.141263 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2231ce7c-c5e9-40b5-a701-7d447c94a42b service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:19.148019 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2231ce7c-c5e9-40b5-a701-7d447c94a42b service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:19.148507 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2231ce7c-c5e9-40b5-a701-7d447c94a42b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:19.188379 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2231ce7c-c5e9-40b5-a701-7d447c94a42b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:19.190481 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2231ce7c-c5e9-40b5-a701-7d447c94a42b service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:19.218420 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 817/1649] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:46:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:46:19.541186 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3111a96-7d89-472e-abf3-31e0a4050b5f service neutron] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:19.542418 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-e3111a96-7d89-472e-abf3-31e0a4050b5f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:19.558805 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3111a96-7d89-472e-abf3-31e0a4050b5f service neutron] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:19.565390 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cc47c6-49c9-406f-b27c-33adf979357a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:19.566999 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cc47c6-49c9-406f-b27c-33adf979357a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:19.566999 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cc47c6-49c9-406f-b27c-33adf979357a admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:19.566999 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cc47c6-49c9-406f-b27c-33adf979357a admin admin] PATH_INFO: `/v3/users/23b1773e3f4e47aca50c664edf55882e` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:19.568725 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cc47c6-49c9-406f-b27c-33adf979357a admin admin] RBAC: Authorizing `identity:delete_user(user_id=23b1773e3f4e47aca50c664edf55882e)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:19.571782 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cc47c6-49c9-406f-b27c-33adf979357a admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:19.650587 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-f6cc47c6-49c9-406f-b27c-33adf979357a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23b1773e3f4e47aca50c664edf55882e'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:19.661923 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-f6cc47c6-49c9-406f-b27c-33adf979357a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23b1773e3f4e47aca50c664edf55882e'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:19.670628 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-f6cc47c6-49c9-406f-b27c-33adf979357a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23b1773e3f4e47aca50c664edf55882e'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:19.671681 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-f6cc47c6-49c9-406f-b27c-33adf979357a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23b1773e3f4e47aca50c664edf55882e'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:19.672544 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-f6cc47c6-49c9-406f-b27c-33adf979357a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23b1773e3f4e47aca50c664edf55882e'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:19.702299 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 833/1650] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 13:46:19 2022] DELETE /identity/v3/users/23b1773e3f4e47aca50c664edf55882e => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:46:19.708681 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896a1104-bcb3-4cb5-a07b-6e3dfb78dae8 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:19.709728 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896a1104-bcb3-4cb5-a07b-6e3dfb78dae8 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:19.710235 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896a1104-bcb3-4cb5-a07b-6e3dfb78dae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:19.741168 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-896a1104-bcb3-4cb5-a07b-6e3dfb78dae8 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:19.743933 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-896a1104-bcb3-4cb5-a07b-6e3dfb78dae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:19.882640 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 818/1651] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:46:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 13:46:19.884252 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cc47c6-49c9-406f-b27c-33adf979357a admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:19.891772 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-f6cc47c6-49c9-406f-b27c-33adf979357a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:19.972887 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:19.988199 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cc47c6-49c9-406f-b27c-33adf979357a admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:19.998295 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4a947b-12f7-4427-8d0c-44bba1465267 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:19.999720 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4a947b-12f7-4427-8d0c-44bba1465267 service neutron] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:20.000426 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4a947b-12f7-4427-8d0c-44bba1465267 service neutron] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:20.000679 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4a947b-12f7-4427-8d0c-44bba1465267 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:20.013911 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4a947b-12f7-4427-8d0c-44bba1465267 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:20.015669 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4a947b-12f7-4427-8d0c-44bba1465267 service neutron] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:20.030478 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 834/1652] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:46:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:46:20.225985 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896a1104-bcb3-4cb5-a07b-6e3dfb78dae8 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:20.227440 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-896a1104-bcb3-4cb5-a07b-6e3dfb78dae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:20.252755 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896a1104-bcb3-4cb5-a07b-6e3dfb78dae8 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:20.260179 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aecfdbc-7e89-4ce1-b0a3-b73b8d0d6fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:20.260997 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aecfdbc-7e89-4ce1-b0a3-b73b8d0d6fae admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:20.261556 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aecfdbc-7e89-4ce1-b0a3-b73b8d0d6fae admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:20.261910 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aecfdbc-7e89-4ce1-b0a3-b73b8d0d6fae admin admin] PATH_INFO: `/v3/projects/48ca92b54ca442c09ecfcfa3af12469a` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:20.264434 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aecfdbc-7e89-4ce1-b0a3-b73b8d0d6fae admin admin] RBAC: Authorizing `identity:delete_project(project_id=48ca92b54ca442c09ecfcfa3af12469a)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:20.296415 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: WARNING py.warnings [None req-5aecfdbc-7e89-4ce1-b0a3-b73b8d0d6fae admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 13:46:20.296415 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: warnings.warn(msg) Jul 26 13:46:20.296415 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: Jul 26 13:46:20.300943 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aecfdbc-7e89-4ce1-b0a3-b73b8d0d6fae admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:20.499310 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4a947b-12f7-4427-8d0c-44bba1465267 service neutron] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:20.502259 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-bc4a947b-12f7-4427-8d0c-44bba1465267 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:20.502390 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-5aecfdbc-7e89-4ce1-b0a3-b73b8d0d6fae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48ca92b54ca442c09ecfcfa3af12469a', 'request_id': 'req-5aecfdbc-7e89-4ce1-b0a3-b73b8d0d6fae'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:20.538219 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-5aecfdbc-7e89-4ce1-b0a3-b73b8d0d6fae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48ca92b54ca442c09ecfcfa3af12469a', 'request_id': 'req-5aecfdbc-7e89-4ce1-b0a3-b73b8d0d6fae'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:20.556519 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-5aecfdbc-7e89-4ce1-b0a3-b73b8d0d6fae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48ca92b54ca442c09ecfcfa3af12469a', 'request_id': 'req-5aecfdbc-7e89-4ce1-b0a3-b73b8d0d6fae'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:20.562436 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4a947b-12f7-4427-8d0c-44bba1465267 service neutron] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:20.576372 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-5aecfdbc-7e89-4ce1-b0a3-b73b8d0d6fae admin admin] The token cache is being invalidate because project 48ca92b54ca442c09ecfcfa3af12469a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 13:46:20.576695 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-5aecfdbc-7e89-4ce1-b0a3-b73b8d0d6fae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:20.577339 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-5aecfdbc-7e89-4ce1-b0a3-b73b8d0d6fae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:20.589969 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 819/1653] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 13:46:20 2022] DELETE /identity/v3/projects/48ca92b54ca442c09ecfcfa3af12469a => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:46:20.591097 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aecfdbc-7e89-4ce1-b0a3-b73b8d0d6fae admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:20.592559 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-5aecfdbc-7e89-4ce1-b0a3-b73b8d0d6fae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:20.721610 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddd5b72-e5a8-44c8-a1f7-67d43f334928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:20.722298 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddd5b72-e5a8-44c8-a1f7-67d43f334928 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:20.722585 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddd5b72-e5a8-44c8-a1f7-67d43f334928 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:20.722864 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddd5b72-e5a8-44c8-a1f7-67d43f334928 admin admin] PATH_INFO: `/v3/projects/560db1a629c4491abbc1742ddcec0747` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:20.723827 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aecfdbc-7e89-4ce1-b0a3-b73b8d0d6fae admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:20.724303 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddd5b72-e5a8-44c8-a1f7-67d43f334928 admin admin] RBAC: Authorizing `identity:delete_project(project_id=560db1a629c4491abbc1742ddcec0747)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:20.726722 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddd5b72-e5a8-44c8-a1f7-67d43f334928 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:20.736164 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f63988d-0b5d-4be7-8d2f-f7f579f00275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:20.736858 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f63988d-0b5d-4be7-8d2f-f7f579f00275 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:20.737145 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f63988d-0b5d-4be7-8d2f-f7f579f00275 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:20.737427 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f63988d-0b5d-4be7-8d2f-f7f579f00275 admin admin] PATH_INFO: `/v3/users/149d925dcd354812b3e0dbfdfdb87647` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:20.738759 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f63988d-0b5d-4be7-8d2f-f7f579f00275 admin admin] RBAC: Authorizing `identity:delete_user(user_id=149d925dcd354812b3e0dbfdfdb87647)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:20.740747 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f63988d-0b5d-4be7-8d2f-f7f579f00275 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:20.759033 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:20.842229 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-6ddd5b72-e5a8-44c8-a1f7-67d43f334928 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '560db1a629c4491abbc1742ddcec0747', 'request_id': 'req-6ddd5b72-e5a8-44c8-a1f7-67d43f334928'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:20.863801 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-0f63988d-0b5d-4be7-8d2f-f7f579f00275 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '149d925dcd354812b3e0dbfdfdb87647'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:20.866740 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-6ddd5b72-e5a8-44c8-a1f7-67d43f334928 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '560db1a629c4491abbc1742ddcec0747', 'request_id': 'req-6ddd5b72-e5a8-44c8-a1f7-67d43f334928'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:20.866904 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-0f63988d-0b5d-4be7-8d2f-f7f579f00275 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '149d925dcd354812b3e0dbfdfdb87647'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:20.877200 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-6ddd5b72-e5a8-44c8-a1f7-67d43f334928 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '560db1a629c4491abbc1742ddcec0747', 'request_id': 'req-6ddd5b72-e5a8-44c8-a1f7-67d43f334928'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:20.891726 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-0f63988d-0b5d-4be7-8d2f-f7f579f00275 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '149d925dcd354812b3e0dbfdfdb87647'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:20.895490 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-6ddd5b72-e5a8-44c8-a1f7-67d43f334928 admin admin] The token cache is being invalidate because project 560db1a629c4491abbc1742ddcec0747 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86933) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 13:46:20.895980 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-0f63988d-0b5d-4be7-8d2f-f7f579f00275 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '149d925dcd354812b3e0dbfdfdb87647'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:20.896089 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-6ddd5b72-e5a8-44c8-a1f7-67d43f334928 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:20.896567 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-6ddd5b72-e5a8-44c8-a1f7-67d43f334928 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:20.900021 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 835/1654] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 13:46:20 2022] DELETE /identity/v3/projects/560db1a629c4491abbc1742ddcec0747 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 13:46:20.904732 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddd5b72-e5a8-44c8-a1f7-67d43f334928 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:20.909028 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-6ddd5b72-e5a8-44c8-a1f7-67d43f334928 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:20.930985 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-0f63988d-0b5d-4be7-8d2f-f7f579f00275 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '149d925dcd354812b3e0dbfdfdb87647'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:20.963481 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 820/1655] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 13:46:20 2022] DELETE /identity/v3/users/149d925dcd354812b3e0dbfdfdb87647 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 13:46:21.088637 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddd5b72-e5a8-44c8-a1f7-67d43f334928 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:21.142828 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:21.218400 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1118215-8588-4f58-a032-ddf48056ff03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:21.219095 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1118215-8588-4f58-a032-ddf48056ff03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:21.219383 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1118215-8588-4f58-a032-ddf48056ff03 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:21.219663 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1118215-8588-4f58-a032-ddf48056ff03 admin admin] PATH_INFO: `/v3/users/fa56661a48f046e3918444fc71111689` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:21.221730 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1118215-8588-4f58-a032-ddf48056ff03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa56661a48f046e3918444fc71111689)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:21.223684 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1118215-8588-4f58-a032-ddf48056ff03 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:21.340611 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-a1118215-8588-4f58-a032-ddf48056ff03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa56661a48f046e3918444fc71111689'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:21.367757 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f63988d-0b5d-4be7-8d2f-f7f579f00275 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:21.368636 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-a1118215-8588-4f58-a032-ddf48056ff03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa56661a48f046e3918444fc71111689'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:21.369750 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-0f63988d-0b5d-4be7-8d2f-f7f579f00275 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:21.380310 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-a1118215-8588-4f58-a032-ddf48056ff03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa56661a48f046e3918444fc71111689'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:21.381010 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-a1118215-8588-4f58-a032-ddf48056ff03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa56661a48f046e3918444fc71111689'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:21.382909 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-a1118215-8588-4f58-a032-ddf48056ff03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa56661a48f046e3918444fc71111689'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:21.387167 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f63988d-0b5d-4be7-8d2f-f7f579f00275 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:21.391706 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad70cb1-38a2-4796-967f-2c18c25cb4e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:21.392487 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad70cb1-38a2-4796-967f-2c18c25cb4e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:21.393009 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad70cb1-38a2-4796-967f-2c18c25cb4e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:21.393403 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad70cb1-38a2-4796-967f-2c18c25cb4e2 admin admin] PATH_INFO: `/v3/projects/2842eeced6c14c8dbe84359801ac5fdb` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:21.395247 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad70cb1-38a2-4796-967f-2c18c25cb4e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2842eeced6c14c8dbe84359801ac5fdb)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:21.397130 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad70cb1-38a2-4796-967f-2c18c25cb4e2 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:21.434781 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 836/1656] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 13:46:20 2022] DELETE /identity/v3/users/fa56661a48f046e3918444fc71111689 => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 13:46:21.534013 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-4ad70cb1-38a2-4796-967f-2c18c25cb4e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2842eeced6c14c8dbe84359801ac5fdb', 'request_id': 'req-4ad70cb1-38a2-4796-967f-2c18c25cb4e2'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:21.543796 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-4ad70cb1-38a2-4796-967f-2c18c25cb4e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2842eeced6c14c8dbe84359801ac5fdb', 'request_id': 'req-4ad70cb1-38a2-4796-967f-2c18c25cb4e2'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:21.561515 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-4ad70cb1-38a2-4796-967f-2c18c25cb4e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2842eeced6c14c8dbe84359801ac5fdb', 'request_id': 'req-4ad70cb1-38a2-4796-967f-2c18c25cb4e2'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:21.583232 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-4ad70cb1-38a2-4796-967f-2c18c25cb4e2 admin admin] The token cache is being invalidate because project 2842eeced6c14c8dbe84359801ac5fdb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 13:46:21.583838 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-4ad70cb1-38a2-4796-967f-2c18c25cb4e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:21.585001 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-4ad70cb1-38a2-4796-967f-2c18c25cb4e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:21.593024 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 821/1657] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 13:46:21 2022] DELETE /identity/v3/projects/2842eeced6c14c8dbe84359801ac5fdb => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:46:21.594073 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e48451-5c17-4a71-8010-0f19fef6d0c7 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:21.594461 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e48451-5c17-4a71-8010-0f19fef6d0c7 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:21.594710 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e48451-5c17-4a71-8010-0f19fef6d0c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:21.629959 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-66e48451-5c17-4a71-8010-0f19fef6d0c7 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:21.630998 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-66e48451-5c17-4a71-8010-0f19fef6d0c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:21.745854 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 837/1658] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:46:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:46:21.754393 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34eaf21-6a46-4acc-b531-326844223bb6 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:21.754876 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34eaf21-6a46-4acc-b531-326844223bb6 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:21.754944 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34eaf21-6a46-4acc-b531-326844223bb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:21.817286 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-e34eaf21-6a46-4acc-b531-326844223bb6 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:21.819304 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-e34eaf21-6a46-4acc-b531-326844223bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:21.835957 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 822/1659] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:46:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:46:21.840154 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e48451-5c17-4a71-8010-0f19fef6d0c7 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:21.862831 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e48451-5c17-4a71-8010-0f19fef6d0c7 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:21.879847 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc7c66b-74fe-4b49-bb31-47dbde86d412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:21.880515 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc7c66b-74fe-4b49-bb31-47dbde86d412 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:21.880800 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc7c66b-74fe-4b49-bb31-47dbde86d412 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:21.881741 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc7c66b-74fe-4b49-bb31-47dbde86d412 admin admin] PATH_INFO: `/v3/domains` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:21.882870 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc7c66b-74fe-4b49-bb31-47dbde86d412 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:46:21.883189 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc7c66b-74fe-4b49-bb31-47dbde86d412 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:21.894029 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc7c66b-74fe-4b49-bb31-47dbde86d412 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:21.915915 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 838/1660] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Jul 26 13:46:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:46:21.920396 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34eaf21-6a46-4acc-b531-326844223bb6 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:21.938095 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34eaf21-6a46-4acc-b531-326844223bb6 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:21.958714 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5dc3b5-e500-4fc7-b3f7-f965991c7b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:21.959519 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5dc3b5-e500-4fc7-b3f7-f965991c7b45 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:21.959902 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5dc3b5-e500-4fc7-b3f7-f965991c7b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:21.960272 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5dc3b5-e500-4fc7-b3f7-f965991c7b45 admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:21.961051 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5dc3b5-e500-4fc7-b3f7-f965991c7b45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:21.965949 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5dc3b5-e500-4fc7-b3f7-f965991c7b45 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:22.029545 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 823/1661] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 13:46:21 2022] POST /identity/v3/projects => generated 405 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:46:22.036556 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc7c66b-74fe-4b49-bb31-47dbde86d412 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:22.062767 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc7c66b-74fe-4b49-bb31-47dbde86d412 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:22.146259 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:22.248540 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613e0fd4-8f0d-4135-b692-9a8cd689aa93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:22.249204 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613e0fd4-8f0d-4135-b692-9a8cd689aa93 admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:22.249556 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613e0fd4-8f0d-4135-b692-9a8cd689aa93 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:22.249841 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613e0fd4-8f0d-4135-b692-9a8cd689aa93 admin admin] PATH_INFO: `/v3/users` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:22.250531 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613e0fd4-8f0d-4135-b692-9a8cd689aa93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:22.252554 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613e0fd4-8f0d-4135-b692-9a8cd689aa93 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:22.332164 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 839/1662] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 13:46:22 2022] POST /identity/v3/users => generated 303 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:46:22.336232 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5dc3b5-e500-4fc7-b3f7-f965991c7b45 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:22.353776 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5dc3b5-e500-4fc7-b3f7-f965991c7b45 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:22.378279 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75a6e67-a1fc-4354-9f0d-11ccab0773e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:22.379532 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75a6e67-a1fc-4354-9f0d-11ccab0773e4 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:22.379830 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75a6e67-a1fc-4354-9f0d-11ccab0773e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:22.380110 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75a6e67-a1fc-4354-9f0d-11ccab0773e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:22.380834 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75a6e67-a1fc-4354-9f0d-11ccab0773e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:22.386037 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75a6e67-a1fc-4354-9f0d-11ccab0773e4 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:22.414867 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 824/1663] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:46:22 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:46:22.416266 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613e0fd4-8f0d-4135-b692-9a8cd689aa93 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:22.427699 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613e0fd4-8f0d-4135-b692-9a8cd689aa93 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:22.443974 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc22617-2418-4309-bb75-dbd8cc260cee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:22.447567 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc22617-2418-4309-bb75-dbd8cc260cee admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:22.447567 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc22617-2418-4309-bb75-dbd8cc260cee admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:22.447567 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc22617-2418-4309-bb75-dbd8cc260cee admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:22.447567 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc22617-2418-4309-bb75-dbd8cc260cee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:22.448356 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc22617-2418-4309-bb75-dbd8cc260cee admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:22.469181 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 840/1664] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:46:22 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:46:22.471024 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75a6e67-a1fc-4354-9f0d-11ccab0773e4 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:22.487148 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75a6e67-a1fc-4354-9f0d-11ccab0773e4 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:22.493403 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea605770-8e65-496d-a2fa-aa188add9ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:22.494534 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea605770-8e65-496d-a2fa-aa188add9ead admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:22.494913 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea605770-8e65-496d-a2fa-aa188add9ead admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:22.508598 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea605770-8e65-496d-a2fa-aa188add9ead admin admin] PATH_INFO: `/v3/projects/c057f946baaf43c1a6d147d846418604/users/66217dc1b3f84d82a5300ee1ff983da4/roles/759f389b221049c4b01faf3b984cd8d7` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:22.526962 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea605770-8e65-496d-a2fa-aa188add9ead admin admin] RBAC: Authorizing `identity:create_grant(project_id=c057f946baaf43c1a6d147d846418604, user_id=66217dc1b3f84d82a5300ee1ff983da4, role_id=759f389b221049c4b01faf3b984cd8d7)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:22.530788 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea605770-8e65-496d-a2fa-aa188add9ead admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:22.560537 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 825/1665] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 13:46:22 2022] PUT /identity/v3/projects/c057f946baaf43c1a6d147d846418604/users/66217dc1b3f84d82a5300ee1ff983da4/roles/759f389b221049c4b01faf3b984cd8d7 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 13:46:22.566469 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e3fae5-d0c7-4e24-a488-d2ea6d67ae21 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:22.567262 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e3fae5-d0c7-4e24-a488-d2ea6d67ae21 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:22.567903 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e3fae5-d0c7-4e24-a488-d2ea6d67ae21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:22.602278 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-73e3fae5-d0c7-4e24-a488-d2ea6d67ae21 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:22.603847 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-73e3fae5-d0c7-4e24-a488-d2ea6d67ae21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:22.679145 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 841/1666] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:46:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:46:22.711640 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea605770-8e65-496d-a2fa-aa188add9ead admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:22.714707 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-ea605770-8e65-496d-a2fa-aa188add9ead admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:22.728250 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=79,keystone:DELETE=17,keystone:INSERT=4 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:22.793775 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e3fae5-d0c7-4e24-a488-d2ea6d67ae21 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:22.799979 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-73e3fae5-d0c7-4e24-a488-d2ea6d67ae21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:22.836317 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e3fae5-d0c7-4e24-a488-d2ea6d67ae21 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:22.847239 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea605770-8e65-496d-a2fa-aa188add9ead admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:22.849728 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8a9ba0-e353-4050-ba62-77104323ec3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:22.851037 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8a9ba0-e353-4050-ba62-77104323ec3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:22.851465 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8a9ba0-e353-4050-ba62-77104323ec3b admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:22.853606 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8a9ba0-e353-4050-ba62-77104323ec3b admin admin] PATH_INFO: `/v3/projects/54f5e370c14a44f882b2f8e2dbbb781e` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:22.857107 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8a9ba0-e353-4050-ba62-77104323ec3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=54f5e370c14a44f882b2f8e2dbbb781e)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:22.858546 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a8cd53-0580-48e3-a33f-de433651f291 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:22.859395 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a8cd53-0580-48e3-a33f-de433651f291 service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:22.859937 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a8cd53-0580-48e3-a33f-de433651f291 service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:22.860306 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a8cd53-0580-48e3-a33f-de433651f291 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:22.865778 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8a9ba0-e353-4050-ba62-77104323ec3b admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:22.884995 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a8cd53-0580-48e3-a33f-de433651f291 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:22.886729 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a8cd53-0580-48e3-a33f-de433651f291 service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:22.912160 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 826/1667] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:46:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:46:23.011333 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-5d8a9ba0-e353-4050-ba62-77104323ec3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54f5e370c14a44f882b2f8e2dbbb781e', 'request_id': 'req-5d8a9ba0-e353-4050-ba62-77104323ec3b'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:23.049316 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-5d8a9ba0-e353-4050-ba62-77104323ec3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54f5e370c14a44f882b2f8e2dbbb781e', 'request_id': 'req-5d8a9ba0-e353-4050-ba62-77104323ec3b'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:23.063915 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-5d8a9ba0-e353-4050-ba62-77104323ec3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54f5e370c14a44f882b2f8e2dbbb781e', 'request_id': 'req-5d8a9ba0-e353-4050-ba62-77104323ec3b'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:23.085605 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-5d8a9ba0-e353-4050-ba62-77104323ec3b admin admin] The token cache is being invalidate because project 54f5e370c14a44f882b2f8e2dbbb781e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86933) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 13:46:23.085877 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-5d8a9ba0-e353-4050-ba62-77104323ec3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:23.086571 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-5d8a9ba0-e353-4050-ba62-77104323ec3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:23.088483 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 842/1668] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 13:46:22 2022] DELETE /identity/v3/projects/54f5e370c14a44f882b2f8e2dbbb781e => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:46:23.093611 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711c8158-9526-474b-866b-7540b4a7919c None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:23.093906 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711c8158-9526-474b-866b-7540b4a7919c None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:23.094187 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711c8158-9526-474b-866b-7540b4a7919c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:23.135376 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-711c8158-9526-474b-866b-7540b4a7919c None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:23.136606 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-711c8158-9526-474b-866b-7540b4a7919c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:23.296309 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 827/1669] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:46:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:46:23.296778 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e629a8-0144-4d40-ab98-96855e2c2ab0 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:23.297293 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e629a8-0144-4d40-ab98-96855e2c2ab0 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:23.297394 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e629a8-0144-4d40-ab98-96855e2c2ab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:23.345919 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-e8e629a8-0144-4d40-ab98-96855e2c2ab0 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:23.347519 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-e8e629a8-0144-4d40-ab98-96855e2c2ab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:23.366381 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711c8158-9526-474b-866b-7540b4a7919c None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:23.385571 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 843/1670] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:46:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:46:23.389758 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711c8158-9526-474b-866b-7540b4a7919c None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:23.400669 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc23831-030c-4b19-91eb-7ce295163785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:23.402969 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc23831-030c-4b19-91eb-7ce295163785 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:23.403266 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc23831-030c-4b19-91eb-7ce295163785 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:23.403547 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc23831-030c-4b19-91eb-7ce295163785 admin admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:23.404276 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc23831-030c-4b19-91eb-7ce295163785 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:46:23.404681 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc23831-030c-4b19-91eb-7ce295163785 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:23.406624 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc23831-030c-4b19-91eb-7ce295163785 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:23.447967 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 828/1671] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Jul 26 13:46:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:46:23.448154 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e629a8-0144-4d40-ab98-96855e2c2ab0 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:23.458576 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e629a8-0144-4d40-ab98-96855e2c2ab0 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:23.491593 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3b60a7-2ffd-4bb8-95d3-e3dd05e85664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:23.492379 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3b60a7-2ffd-4bb8-95d3-e3dd05e85664 admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:23.492726 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3b60a7-2ffd-4bb8-95d3-e3dd05e85664 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:23.493007 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3b60a7-2ffd-4bb8-95d3-e3dd05e85664 admin admin] PATH_INFO: `/v3/projects` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:23.493707 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3b60a7-2ffd-4bb8-95d3-e3dd05e85664 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:23.500206 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3b60a7-2ffd-4bb8-95d3-e3dd05e85664 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:23.587301 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 844/1672] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 13:46:23 2022] POST /identity/v3/projects => generated 371 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:46:23.592113 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc23831-030c-4b19-91eb-7ce295163785 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:23.608638 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc23831-030c-4b19-91eb-7ce295163785 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:23.723965 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a87d6f-9dbc-46fa-8339-baf976627dab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:23.724481 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a87d6f-9dbc-46fa-8339-baf976627dab admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:23.724842 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a87d6f-9dbc-46fa-8339-baf976627dab admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:23.725190 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a87d6f-9dbc-46fa-8339-baf976627dab admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:23.725974 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a87d6f-9dbc-46fa-8339-baf976627dab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:23.728814 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a87d6f-9dbc-46fa-8339-baf976627dab admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:23.769010 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3b60a7-2ffd-4bb8-95d3-e3dd05e85664 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:23.791558 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 829/1673] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 13:46:23 2022] POST /identity/v3/users => generated 286 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:46:23.805679 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3b60a7-2ffd-4bb8-95d3-e3dd05e85664 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:23.823555 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d2f0eb-045a-43d2-b90d-44d8677a0718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:23.828670 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d2f0eb-045a-43d2-b90d-44d8677a0718 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:23.830477 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d2f0eb-045a-43d2-b90d-44d8677a0718 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:23.830867 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d2f0eb-045a-43d2-b90d-44d8677a0718 admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:23.833790 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d2f0eb-045a-43d2-b90d-44d8677a0718 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:23.841189 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d2f0eb-045a-43d2-b90d-44d8677a0718 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:23.858715 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:23.879579 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 845/1674] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:46:23 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 13:46:23.881617 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a87d6f-9dbc-46fa-8339-baf976627dab admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:23.912056 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a87d6f-9dbc-46fa-8339-baf976627dab admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:23.936791 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86653e44-e570-4204-ac32-d5525bcfe606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:23.937668 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86653e44-e570-4204-ac32-d5525bcfe606 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:23.938080 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86653e44-e570-4204-ac32-d5525bcfe606 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:23.938458 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86653e44-e570-4204-ac32-d5525bcfe606 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:23.941130 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86653e44-e570-4204-ac32-d5525bcfe606 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:23.943326 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86653e44-e570-4204-ac32-d5525bcfe606 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:23.973807 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 830/1675] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:46:23 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:46:23.974797 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d2f0eb-045a-43d2-b90d-44d8677a0718 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:23.988334 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d2f0eb-045a-43d2-b90d-44d8677a0718 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:24.003363 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f7c643-a68c-4dc0-81f9-6183c0a2eff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:24.004445 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f7c643-a68c-4dc0-81f9-6183c0a2eff7 admin admin] REQUEST_METHOD: `PUT` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:24.004900 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f7c643-a68c-4dc0-81f9-6183c0a2eff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:24.005385 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f7c643-a68c-4dc0-81f9-6183c0a2eff7 admin admin] PATH_INFO: `/v3/projects/73efb3b87fed4f1c9ce8b5eca4d2dd0f/users/b8d66515212945e3bf336827697bcd12/roles/759f389b221049c4b01faf3b984cd8d7` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:24.051544 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f7c643-a68c-4dc0-81f9-6183c0a2eff7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73efb3b87fed4f1c9ce8b5eca4d2dd0f, user_id=b8d66515212945e3bf336827697bcd12, role_id=759f389b221049c4b01faf3b984cd8d7)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:24.060555 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f7c643-a68c-4dc0-81f9-6183c0a2eff7 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:24.090498 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 846/1676] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 13:46:23 2022] PUT /identity/v3/projects/73efb3b87fed4f1c9ce8b5eca4d2dd0f/users/b8d66515212945e3bf336827697bcd12/roles/759f389b221049c4b01faf3b984cd8d7 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:46:24.104444 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719b43ba-4ced-49a2-8bb5-ee7ad51fcdbf None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:24.104740 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719b43ba-4ced-49a2-8bb5-ee7ad51fcdbf None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:24.105019 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719b43ba-4ced-49a2-8bb5-ee7ad51fcdbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:24.151371 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:24.156695 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-719b43ba-4ced-49a2-8bb5-ee7ad51fcdbf None None] MFA Rules not processed for user `b8d66515212945e3bf336827697bcd12`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:24.159321 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-719b43ba-4ced-49a2-8bb5-ee7ad51fcdbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:24.308182 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 831/1677] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:46:24 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:46:24.312910 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f7c643-a68c-4dc0-81f9-6183c0a2eff7 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:24.428484 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719b43ba-4ced-49a2-8bb5-ee7ad51fcdbf None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:24.430428 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-719b43ba-4ced-49a2-8bb5-ee7ad51fcdbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:24.474865 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f7c643-a68c-4dc0-81f9-6183c0a2eff7 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:24.481122 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e00d47b-629e-4944-9f6e-2f2525c83d4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:24.497136 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e00d47b-629e-4944-9f6e-2f2525c83d4f admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:24.498266 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e00d47b-629e-4944-9f6e-2f2525c83d4f admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:24.499123 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e00d47b-629e-4944-9f6e-2f2525c83d4f admin admin] PATH_INFO: `/v3/projects` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:24.501881 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e00d47b-629e-4944-9f6e-2f2525c83d4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:24.507568 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e00d47b-629e-4944-9f6e-2f2525c83d4f admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:24.574049 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 847/1678] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 13:46:24 2022] POST /identity/v3/projects => generated 371 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:46:24.575251 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e00d47b-629e-4944-9f6e-2f2525c83d4f admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:24.729204 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719b43ba-4ced-49a2-8bb5-ee7ad51fcdbf None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:24.784978 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e00d47b-629e-4944-9f6e-2f2525c83d4f admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:24.803679 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c0289e-0fa1-4444-bb55-2d9b552870d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:24.805903 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c0289e-0fa1-4444-bb55-2d9b552870d0 admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:24.807042 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c0289e-0fa1-4444-bb55-2d9b552870d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:24.807335 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c0289e-0fa1-4444-bb55-2d9b552870d0 admin admin] PATH_INFO: `/v3/users` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:24.809532 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c0289e-0fa1-4444-bb55-2d9b552870d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:24.815800 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c0289e-0fa1-4444-bb55-2d9b552870d0 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:24.872715 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 848/1679] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 13:46:24 2022] POST /identity/v3/users => generated 292 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:46:24.874686 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c0289e-0fa1-4444-bb55-2d9b552870d0 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:24.903856 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c0289e-0fa1-4444-bb55-2d9b552870d0 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:24.918979 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a72e8c5-c276-49a8-b26d-3363342ec6f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:24.920015 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a72e8c5-c276-49a8-b26d-3363342ec6f0 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:24.921251 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a72e8c5-c276-49a8-b26d-3363342ec6f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:24.922093 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a72e8c5-c276-49a8-b26d-3363342ec6f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:24.922892 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a72e8c5-c276-49a8-b26d-3363342ec6f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:24.929240 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a72e8c5-c276-49a8-b26d-3363342ec6f0 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:24.962218 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 849/1680] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:46:24 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:46:24.965509 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a72e8c5-c276-49a8-b26d-3363342ec6f0 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:24.992978 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a72e8c5-c276-49a8-b26d-3363342ec6f0 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:24.999606 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0ca126-39a4-49c6-9325-27e3ca6492e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:25.000290 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ca126-39a4-49c6-9325-27e3ca6492e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:25.000579 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ca126-39a4-49c6-9325-27e3ca6492e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:25.000857 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ca126-39a4-49c6-9325-27e3ca6492e9 admin admin] PATH_INFO: `/v3/projects/d194af8db3114da7b5f2e5741f428461` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:25.006711 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d14cd34-65ce-4438-a1f2-e88b39534139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:25.007564 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d14cd34-65ce-4438-a1f2-e88b39534139 admin admin] REQUEST_METHOD: `PUT` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:25.008285 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d14cd34-65ce-4438-a1f2-e88b39534139 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:25.009024 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d14cd34-65ce-4438-a1f2-e88b39534139 admin admin] PATH_INFO: `/v3/projects/52de86fb9f0d4b67afa32693f2ae242f/users/bc27cf21a70941559d3f89ca36cae5e1/roles/6e0201a9a6484f7d8b112b1eafff70bd` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:25.009928 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0ca126-39a4-49c6-9325-27e3ca6492e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d194af8db3114da7b5f2e5741f428461)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:25.016148 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0ca126-39a4-49c6-9325-27e3ca6492e9 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:25.056532 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d14cd34-65ce-4438-a1f2-e88b39534139 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52de86fb9f0d4b67afa32693f2ae242f, user_id=bc27cf21a70941559d3f89ca36cae5e1, role_id=6e0201a9a6484f7d8b112b1eafff70bd)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:25.059582 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d14cd34-65ce-4438-a1f2-e88b39534139 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:25.091006 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 850/1681] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 13:46:24 2022] PUT /identity/v3/projects/52de86fb9f0d4b67afa32693f2ae242f/users/bc27cf21a70941559d3f89ca36cae5e1/roles/6e0201a9a6484f7d8b112b1eafff70bd => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:46:25.096776 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d14cd34-65ce-4438-a1f2-e88b39534139 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:25.128671 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:25.169174 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-1a0ca126-39a4-49c6-9325-27e3ca6492e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd194af8db3114da7b5f2e5741f428461', 'request_id': 'req-1a0ca126-39a4-49c6-9325-27e3ca6492e9'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:25.174842 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:25.191416 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-1a0ca126-39a4-49c6-9325-27e3ca6492e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd194af8db3114da7b5f2e5741f428461', 'request_id': 'req-1a0ca126-39a4-49c6-9325-27e3ca6492e9'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:25.210052 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-1a0ca126-39a4-49c6-9325-27e3ca6492e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd194af8db3114da7b5f2e5741f428461', 'request_id': 'req-1a0ca126-39a4-49c6-9325-27e3ca6492e9'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:25.212261 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d14cd34-65ce-4438-a1f2-e88b39534139 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:25.240099 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-1a0ca126-39a4-49c6-9325-27e3ca6492e9 admin admin] The token cache is being invalidate because project d194af8db3114da7b5f2e5741f428461 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 13:46:25.240405 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-1a0ca126-39a4-49c6-9325-27e3ca6492e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:25.244082 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-1a0ca126-39a4-49c6-9325-27e3ca6492e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:25.252311 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 832/1682] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 13:46:24 2022] DELETE /identity/v3/projects/d194af8db3114da7b5f2e5741f428461 => generated 0 bytes in 820 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 13:46:25.256594 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b1bff2-8f0b-4dad-881a-acb565c58dbc None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:25.256893 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b1bff2-8f0b-4dad-881a-acb565c58dbc None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:25.259112 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b1bff2-8f0b-4dad-881a-acb565c58dbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:25.286895 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4309bec-f8d7-4fee-a66c-8d257c3d6ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:25.287677 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4309bec-f8d7-4fee-a66c-8d257c3d6ed4 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:25.288017 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4309bec-f8d7-4fee-a66c-8d257c3d6ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:25.288357 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4309bec-f8d7-4fee-a66c-8d257c3d6ed4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:25.289130 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4309bec-f8d7-4fee-a66c-8d257c3d6ed4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:25.290998 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4309bec-f8d7-4fee-a66c-8d257c3d6ed4 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:25.305855 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 851/1683] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:46:25 2022] GET /identity/v3/roles => generated 1420 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:46:25.309041 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4309bec-f8d7-4fee-a66c-8d257c3d6ed4 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:25.310757 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-03b1bff2-8f0b-4dad-881a-acb565c58dbc None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:25.314492 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-03b1bff2-8f0b-4dad-881a-acb565c58dbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:25.316293 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-c4309bec-f8d7-4fee-a66c-8d257c3d6ed4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:25.339043 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 833/1684] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:46:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:46:25.343809 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4309bec-f8d7-4fee-a66c-8d257c3d6ed4 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:25.348473 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ce3123-baa6-4b3a-8ee3-8f5ca8cd7b0b None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:25.348753 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ce3123-baa6-4b3a-8ee3-8f5ca8cd7b0b None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:25.349059 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ce3123-baa6-4b3a-8ee3-8f5ca8cd7b0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:25.358264 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197b315e-b217-43c5-8b4c-997901bbcb3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:25.358969 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197b315e-b217-43c5-8b4c-997901bbcb3f admin admin] REQUEST_METHOD: `PUT` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:25.359252 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197b315e-b217-43c5-8b4c-997901bbcb3f admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:25.359529 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197b315e-b217-43c5-8b4c-997901bbcb3f admin admin] PATH_INFO: `/v3/projects/52de86fb9f0d4b67afa32693f2ae242f/users/bc27cf21a70941559d3f89ca36cae5e1/roles/6e0201a9a6484f7d8b112b1eafff70bd` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:25.364682 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197b315e-b217-43c5-8b4c-997901bbcb3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=52de86fb9f0d4b67afa32693f2ae242f, user_id=bc27cf21a70941559d3f89ca36cae5e1, role_id=6e0201a9a6484f7d8b112b1eafff70bd)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:25.367357 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197b315e-b217-43c5-8b4c-997901bbcb3f admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:25.405781 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 852/1685] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 13:46:25 2022] PUT /identity/v3/projects/52de86fb9f0d4b67afa32693f2ae242f/users/bc27cf21a70941559d3f89ca36cae5e1/roles/6e0201a9a6484f7d8b112b1eafff70bd => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:46:25.407662 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-36ce3123-baa6-4b3a-8ee3-8f5ca8cd7b0b None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:25.408152 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826a5b26-b866-4b19-9be5-7ec6eaf77cbe None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:25.408441 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826a5b26-b866-4b19-9be5-7ec6eaf77cbe None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:25.408873 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826a5b26-b866-4b19-9be5-7ec6eaf77cbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:25.408972 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-36ce3123-baa6-4b3a-8ee3-8f5ca8cd7b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:25.472967 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-826a5b26-b866-4b19-9be5-7ec6eaf77cbe None None] MFA Rules not processed for user `bc27cf21a70941559d3f89ca36cae5e1`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:25.474239 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-826a5b26-b866-4b19-9be5-7ec6eaf77cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:25.539124 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 834/1686] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:46:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 13:46:25.543552 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ce3123-baa6-4b3a-8ee3-8f5ca8cd7b0b None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:25.577071 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ce3123-baa6-4b3a-8ee3-8f5ca8cd7b0b None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:25.594061 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c300158-369a-4efb-b2ad-f57cc07c737e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:25.595409 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c300158-369a-4efb-b2ad-f57cc07c737e admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:25.595806 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c300158-369a-4efb-b2ad-f57cc07c737e admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:25.596094 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c300158-369a-4efb-b2ad-f57cc07c737e admin admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:25.597989 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c300158-369a-4efb-b2ad-f57cc07c737e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:46:25.598304 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c300158-369a-4efb-b2ad-f57cc07c737e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:25.612119 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c300158-369a-4efb-b2ad-f57cc07c737e admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:25.652293 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 835/1687] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Jul 26 13:46:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:46:25.659366 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c300158-369a-4efb-b2ad-f57cc07c737e admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:25.674236 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 853/1688] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:46:25 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:46:25.684493 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c300158-369a-4efb-b2ad-f57cc07c737e admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:25.689055 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826a5b26-b866-4b19-9be5-7ec6eaf77cbe None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:25.692159 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-826a5b26-b866-4b19-9be5-7ec6eaf77cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:25.695565 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476020d5-f3f4-4436-9f6a-dc61f4cad014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:25.697517 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476020d5-f3f4-4436-9f6a-dc61f4cad014 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:25.697804 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476020d5-f3f4-4436-9f6a-dc61f4cad014 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:25.698086 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476020d5-f3f4-4436-9f6a-dc61f4cad014 admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:25.699095 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476020d5-f3f4-4436-9f6a-dc61f4cad014 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:25.701659 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476020d5-f3f4-4436-9f6a-dc61f4cad014 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:25.788722 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 836/1689] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 13:46:25 2022] POST /identity/v3/projects => generated 381 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:46:25.793226 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476020d5-f3f4-4436-9f6a-dc61f4cad014 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:25.794225 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:25.856048 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826a5b26-b866-4b19-9be5-7ec6eaf77cbe None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:26.003184 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14136bc3-24ae-499e-8b30-e21df854bb65 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:26.005257 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14136bc3-24ae-499e-8b30-e21df854bb65 service cinder] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:26.005571 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14136bc3-24ae-499e-8b30-e21df854bb65 service cinder] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:26.005850 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14136bc3-24ae-499e-8b30-e21df854bb65 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:26.049184 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476020d5-f3f4-4436-9f6a-dc61f4cad014 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:26.065296 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73496e9-a2e2-42a3-8dec-f2de0e14d215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:26.065976 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73496e9-a2e2-42a3-8dec-f2de0e14d215 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:26.066261 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73496e9-a2e2-42a3-8dec-f2de0e14d215 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:26.066542 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73496e9-a2e2-42a3-8dec-f2de0e14d215 admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:26.067221 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73496e9-a2e2-42a3-8dec-f2de0e14d215 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:26.071899 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73496e9-a2e2-42a3-8dec-f2de0e14d215 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:26.119868 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 837/1690] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 13:46:25 2022] POST /identity/v3/users => generated 291 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:46:26.128670 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73496e9-a2e2-42a3-8dec-f2de0e14d215 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:26.143370 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73496e9-a2e2-42a3-8dec-f2de0e14d215 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:26.152386 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082805cd-0aa9-49c3-be76-7bfa2c5f7d88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:26.153302 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082805cd-0aa9-49c3-be76-7bfa2c5f7d88 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:26.154952 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082805cd-0aa9-49c3-be76-7bfa2c5f7d88 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:26.156833 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082805cd-0aa9-49c3-be76-7bfa2c5f7d88 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:26.159346 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082805cd-0aa9-49c3-be76-7bfa2c5f7d88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:26.163971 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082805cd-0aa9-49c3-be76-7bfa2c5f7d88 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:26.198707 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 838/1691] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:46:26 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:46:26.202783 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082805cd-0aa9-49c3-be76-7bfa2c5f7d88 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:26.212611 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14136bc3-24ae-499e-8b30-e21df854bb65 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:26.214399 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14136bc3-24ae-499e-8b30-e21df854bb65 service cinder] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:26.221341 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082805cd-0aa9-49c3-be76-7bfa2c5f7d88 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:26.231843 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03636f0-ca3e-4424-afe5-f10c9c3a5936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:26.232648 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03636f0-ca3e-4424-afe5-f10c9c3a5936 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:26.232783 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03636f0-ca3e-4424-afe5-f10c9c3a5936 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:26.233036 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03636f0-ca3e-4424-afe5-f10c9c3a5936 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:26.233768 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03636f0-ca3e-4424-afe5-f10c9c3a5936 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:26.237782 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03636f0-ca3e-4424-afe5-f10c9c3a5936 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:26.254170 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 854/1692] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:46:25 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 566 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:46:26.276272 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 839/1693] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:46:26 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:46:26.278787 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14136bc3-24ae-499e-8b30-e21df854bb65 service cinder] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:26.299795 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14136bc3-24ae-499e-8b30-e21df854bb65 service cinder] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:26.307003 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e069cc-6c94-437e-9777-d5aae59f1460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:26.308995 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e069cc-6c94-437e-9777-d5aae59f1460 admin admin] REQUEST_METHOD: `PUT` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:26.309804 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e069cc-6c94-437e-9777-d5aae59f1460 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:26.310324 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e069cc-6c94-437e-9777-d5aae59f1460 admin admin] PATH_INFO: `/v3/projects/c447c7416c734b038a91f2921d5da9f6/users/b48f5ad3693743c6b2a1ad330f059d81/roles/759f389b221049c4b01faf3b984cd8d7` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:26.344748 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e069cc-6c94-437e-9777-d5aae59f1460 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c447c7416c734b038a91f2921d5da9f6, user_id=b48f5ad3693743c6b2a1ad330f059d81, role_id=759f389b221049c4b01faf3b984cd8d7)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:26.348922 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e069cc-6c94-437e-9777-d5aae59f1460 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:26.412535 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 855/1694] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 13:46:26 2022] PUT /identity/v3/projects/c447c7416c734b038a91f2921d5da9f6/users/b48f5ad3693743c6b2a1ad330f059d81/roles/759f389b221049c4b01faf3b984cd8d7 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:46:26.412722 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad8ac17-9195-4a8d-b081-1ae636e54d7b None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:26.412828 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad8ac17-9195-4a8d-b081-1ae636e54d7b None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:26.413090 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad8ac17-9195-4a8d-b081-1ae636e54d7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:26.437360 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=92,keystone:DELETE=3,keystone:INSERT=5 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:26.477343 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-bad8ac17-9195-4a8d-b081-1ae636e54d7b None None] MFA Rules not processed for user `b48f5ad3693743c6b2a1ad330f059d81`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:26.478655 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-bad8ac17-9195-4a8d-b081-1ae636e54d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:26.634334 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 840/1695] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:46:26 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:46:26.636555 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e069cc-6c94-437e-9777-d5aae59f1460 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:26.830930 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e069cc-6c94-437e-9777-d5aae59f1460 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:26.860006 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbce98f-f14e-42af-a887-946db2c535f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:26.861197 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbce98f-f14e-42af-a887-946db2c535f1 admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:26.863336 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbce98f-f14e-42af-a887-946db2c535f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:26.863911 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbce98f-f14e-42af-a887-946db2c535f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:26.864761 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbce98f-f14e-42af-a887-946db2c535f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:26.869891 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbce98f-f14e-42af-a887-946db2c535f1 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:26.916631 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:26.979117 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad8ac17-9195-4a8d-b081-1ae636e54d7b None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:26.996410 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 856/1696] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 13:46:26 2022] POST /identity/v3/projects => generated 381 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:46:26.997760 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad8ac17-9195-4a8d-b081-1ae636e54d7b None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:27.186388 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26761ae8-879f-449b-8468-1533ea06be77 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:27.186858 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26761ae8-879f-449b-8468-1533ea06be77 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:27.187456 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26761ae8-879f-449b-8468-1533ea06be77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:27.192350 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0ba57a-4518-4710-af7a-eac46e164be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:27.192933 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0ba57a-4518-4710-af7a-eac46e164be5 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:27.193218 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0ba57a-4518-4710-af7a-eac46e164be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:27.193529 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0ba57a-4518-4710-af7a-eac46e164be5 admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:27.194348 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0ba57a-4518-4710-af7a-eac46e164be5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:27.196290 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0ba57a-4518-4710-af7a-eac46e164be5 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:27.238502 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-26761ae8-879f-449b-8468-1533ea06be77 None None] MFA Rules not processed for user `66217dc1b3f84d82a5300ee1ff983da4`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:27.241098 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-26761ae8-879f-449b-8468-1533ea06be77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:27.248555 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 841/1697] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 13:46:26 2022] POST /identity/v3/users => generated 297 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:46:27.251869 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0ba57a-4518-4710-af7a-eac46e164be5 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:27.271450 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0ba57a-4518-4710-af7a-eac46e164be5 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:27.293788 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838fa3be-e5b4-41bc-888c-fc99990c47e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:27.294754 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838fa3be-e5b4-41bc-888c-fc99990c47e4 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:27.295043 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838fa3be-e5b4-41bc-888c-fc99990c47e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:27.295323 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838fa3be-e5b4-41bc-888c-fc99990c47e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:27.296221 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838fa3be-e5b4-41bc-888c-fc99990c47e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:27.298344 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838fa3be-e5b4-41bc-888c-fc99990c47e4 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:27.348599 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 842/1698] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:46:27 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:46:27.349600 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838fa3be-e5b4-41bc-888c-fc99990c47e4 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:27.367505 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 857/1699] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:46:27 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 13:46:27.370061 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838fa3be-e5b4-41bc-888c-fc99990c47e4 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:27.373422 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26761ae8-879f-449b-8468-1533ea06be77 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:27.378888 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-26761ae8-879f-449b-8468-1533ea06be77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:27.396340 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69d0d95-92dd-4c96-8d91-85839b5cb0b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:27.397148 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69d0d95-92dd-4c96-8d91-85839b5cb0b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:27.397434 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69d0d95-92dd-4c96-8d91-85839b5cb0b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:27.397757 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69d0d95-92dd-4c96-8d91-85839b5cb0b8 admin admin] PATH_INFO: `/v3/projects/8b3c5544eaf548beb951ab4548e6dbd0/users/fb001c7fda1b4bd492c981b7298c7239/roles/6e0201a9a6484f7d8b112b1eafff70bd` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:27.411945 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26761ae8-879f-449b-8468-1533ea06be77 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:27.422176 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03b1cde-f0ee-4cea-9448-9a9be6ea6ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:27.423894 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03b1cde-f0ee-4cea-9448-9a9be6ea6ab6 admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:27.424266 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03b1cde-f0ee-4cea-9448-9a9be6ea6ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:27.424902 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03b1cde-f0ee-4cea-9448-9a9be6ea6ab6 admin admin] PATH_INFO: `/v3/projects` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:27.425755 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03b1cde-f0ee-4cea-9448-9a9be6ea6ab6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:27.431560 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03b1cde-f0ee-4cea-9448-9a9be6ea6ab6 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:27.439373 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69d0d95-92dd-4c96-8d91-85839b5cb0b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b3c5544eaf548beb951ab4548e6dbd0, user_id=fb001c7fda1b4bd492c981b7298c7239, role_id=6e0201a9a6484f7d8b112b1eafff70bd)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:27.442424 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69d0d95-92dd-4c96-8d91-85839b5cb0b8 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:27.472357 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 858/1700] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 13:46:27 2022] POST /identity/v3/projects => generated 403 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:46:27.476130 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 843/1701] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 13:46:27 2022] PUT /identity/v3/projects/8b3c5544eaf548beb951ab4548e6dbd0/users/fb001c7fda1b4bd492c981b7298c7239/roles/6e0201a9a6484f7d8b112b1eafff70bd => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:46:27.476333 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03b1cde-f0ee-4cea-9448-9a9be6ea6ab6 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:27.479820 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69d0d95-92dd-4c96-8d91-85839b5cb0b8 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:27.619778 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03b1cde-f0ee-4cea-9448-9a9be6ea6ab6 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:27.633046 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6419d25-90a4-47c6-a588-ac0e46105d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:27.633743 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6419d25-90a4-47c6-a588-ac0e46105d24 admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:27.634025 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6419d25-90a4-47c6-a588-ac0e46105d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:27.634302 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6419d25-90a4-47c6-a588-ac0e46105d24 admin admin] PATH_INFO: `/v3/users` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:27.634983 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6419d25-90a4-47c6-a588-ac0e46105d24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:27.637552 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6419d25-90a4-47c6-a588-ac0e46105d24 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:27.647789 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:27.681917 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69d0d95-92dd-4c96-8d91-85839b5cb0b8 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:27.686579 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 859/1702] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 13:46:27 2022] POST /identity/v3/users => generated 308 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:46:27.686579 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6419d25-90a4-47c6-a588-ac0e46105d24 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:27.691542 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-a6419d25-90a4-47c6-a588-ac0e46105d24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:27.697624 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d23947-9adb-44f8-ab4a-7602d4eb563f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:27.698213 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d23947-9adb-44f8-ab4a-7602d4eb563f admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:27.698608 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d23947-9adb-44f8-ab4a-7602d4eb563f admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:27.699184 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d23947-9adb-44f8-ab4a-7602d4eb563f admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:27.702239 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d23947-9adb-44f8-ab4a-7602d4eb563f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:27.705782 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d23947-9adb-44f8-ab4a-7602d4eb563f admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:27.719068 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 844/1703] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:46:27 2022] GET /identity/v3/roles => generated 1420 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:46:27.721053 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d23947-9adb-44f8-ab4a-7602d4eb563f admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:27.722179 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6419d25-90a4-47c6-a588-ac0e46105d24 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:27.733852 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d23947-9adb-44f8-ab4a-7602d4eb563f admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:27.736673 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b22b54-cc6f-4e34-a0cd-8287b4f65262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:27.737457 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b22b54-cc6f-4e34-a0cd-8287b4f65262 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:27.737760 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b22b54-cc6f-4e34-a0cd-8287b4f65262 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:27.738162 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b22b54-cc6f-4e34-a0cd-8287b4f65262 admin admin] PATH_INFO: `/v3/users/2b04d57398a44aaba82efa3015ac4983` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:27.740095 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b22b54-cc6f-4e34-a0cd-8287b4f65262 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b04d57398a44aaba82efa3015ac4983)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:27.740325 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629f95dc-661f-4ca3-bc2c-ce6b87cb2803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:27.741099 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629f95dc-661f-4ca3-bc2c-ce6b87cb2803 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:27.741391 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629f95dc-661f-4ca3-bc2c-ce6b87cb2803 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:27.741723 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629f95dc-661f-4ca3-bc2c-ce6b87cb2803 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:27.742054 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b22b54-cc6f-4e34-a0cd-8287b4f65262 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:27.742426 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629f95dc-661f-4ca3-bc2c-ce6b87cb2803 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:27.744804 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629f95dc-661f-4ca3-bc2c-ce6b87cb2803 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:27.773514 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 845/1704] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:46:27 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:46:27.773514 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629f95dc-661f-4ca3-bc2c-ce6b87cb2803 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:27.795292 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629f95dc-661f-4ca3-bc2c-ce6b87cb2803 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:27.803244 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61087cc-a520-4d6b-aad8-0f7bb2c6cd87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:27.803904 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61087cc-a520-4d6b-aad8-0f7bb2c6cd87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:27.804187 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61087cc-a520-4d6b-aad8-0f7bb2c6cd87 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:27.804465 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61087cc-a520-4d6b-aad8-0f7bb2c6cd87 admin admin] PATH_INFO: `/v3/users/b8d66515212945e3bf336827697bcd12` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:27.805772 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61087cc-a520-4d6b-aad8-0f7bb2c6cd87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8d66515212945e3bf336827697bcd12)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:27.812000 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61087cc-a520-4d6b-aad8-0f7bb2c6cd87 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:27.840509 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-82b22b54-cc6f-4e34-a0cd-8287b4f65262 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b04d57398a44aaba82efa3015ac4983'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:27.843350 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:27.857278 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-82b22b54-cc6f-4e34-a0cd-8287b4f65262 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b04d57398a44aaba82efa3015ac4983'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:27.871157 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-82b22b54-cc6f-4e34-a0cd-8287b4f65262 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b04d57398a44aaba82efa3015ac4983'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:27.873574 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-82b22b54-cc6f-4e34-a0cd-8287b4f65262 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b04d57398a44aaba82efa3015ac4983'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:27.873751 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-82b22b54-cc6f-4e34-a0cd-8287b4f65262 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b04d57398a44aaba82efa3015ac4983'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:27.900144 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 860/1705] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 13:46:27 2022] DELETE /identity/v3/users/2b04d57398a44aaba82efa3015ac4983 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:46:27.901155 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b22b54-cc6f-4e34-a0cd-8287b4f65262 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:27.915386 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-82b22b54-cc6f-4e34-a0cd-8287b4f65262 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:27.929855 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-e61087cc-a520-4d6b-aad8-0f7bb2c6cd87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8d66515212945e3bf336827697bcd12'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:27.930886 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-e61087cc-a520-4d6b-aad8-0f7bb2c6cd87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8d66515212945e3bf336827697bcd12'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:27.941859 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-e61087cc-a520-4d6b-aad8-0f7bb2c6cd87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8d66515212945e3bf336827697bcd12'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:27.941859 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-e61087cc-a520-4d6b-aad8-0f7bb2c6cd87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8d66515212945e3bf336827697bcd12'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:27.957162 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-e61087cc-a520-4d6b-aad8-0f7bb2c6cd87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8d66515212945e3bf336827697bcd12'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:27.982680 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 846/1706] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 13:46:27 2022] DELETE /identity/v3/users/b8d66515212945e3bf336827697bcd12 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:46:27.983809 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61087cc-a520-4d6b-aad8-0f7bb2c6cd87 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:27.986767 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-e61087cc-a520-4d6b-aad8-0f7bb2c6cd87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:28.063239 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61087cc-a520-4d6b-aad8-0f7bb2c6cd87 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:28.069056 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc79b32-cc04-44e6-a077-36c4855d9768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:28.070089 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc79b32-cc04-44e6-a077-36c4855d9768 admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:28.070473 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc79b32-cc04-44e6-a077-36c4855d9768 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:28.070928 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc79b32-cc04-44e6-a077-36c4855d9768 admin admin] PATH_INFO: `/v3/projects/63c5b644c2d646478c672a5d7cf6bcc2/users/85f4eb91a7e843e284628d423cf22f7a/roles/6e0201a9a6484f7d8b112b1eafff70bd` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:28.091797 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc79b32-cc04-44e6-a077-36c4855d9768 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63c5b644c2d646478c672a5d7cf6bcc2, user_id=85f4eb91a7e843e284628d423cf22f7a, role_id=6e0201a9a6484f7d8b112b1eafff70bd)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:28.098471 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc79b32-cc04-44e6-a077-36c4855d9768 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:28.120864 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b22b54-cc6f-4e34-a0cd-8287b4f65262 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:28.125062 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 847/1707] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 13:46:27 2022] PUT /identity/v3/projects/63c5b644c2d646478c672a5d7cf6bcc2/users/85f4eb91a7e843e284628d423cf22f7a/roles/6e0201a9a6484f7d8b112b1eafff70bd => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:46:28.125062 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134ee476-42be-4969-8dfe-f588fa75d489 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:28.125062 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134ee476-42be-4969-8dfe-f588fa75d489 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:28.125062 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134ee476-42be-4969-8dfe-f588fa75d489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:28.144628 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-134ee476-42be-4969-8dfe-f588fa75d489 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:28.146149 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-134ee476-42be-4969-8dfe-f588fa75d489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:28.177599 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:28.265334 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 848/1708] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:46:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:46:28.265334 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2996a967-3b7c-4104-aa56-05bdf97b60e6 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:28.265334 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2996a967-3b7c-4104-aa56-05bdf97b60e6 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:28.267910 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc32e73d-9e7e-4276-926b-dd63f6e27c00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:28.268608 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2996a967-3b7c-4104-aa56-05bdf97b60e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:28.277612 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc32e73d-9e7e-4276-926b-dd63f6e27c00 admin admin] REQUEST_METHOD: `PUT` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:28.277612 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc32e73d-9e7e-4276-926b-dd63f6e27c00 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:28.277612 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc32e73d-9e7e-4276-926b-dd63f6e27c00 admin admin] PATH_INFO: `/v3/projects/8b3c5544eaf548beb951ab4548e6dbd0/users/fb001c7fda1b4bd492c981b7298c7239/roles/6e0201a9a6484f7d8b112b1eafff70bd` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:28.285719 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc32e73d-9e7e-4276-926b-dd63f6e27c00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b3c5544eaf548beb951ab4548e6dbd0, user_id=fb001c7fda1b4bd492c981b7298c7239, role_id=6e0201a9a6484f7d8b112b1eafff70bd)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:28.304491 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc32e73d-9e7e-4276-926b-dd63f6e27c00 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:28.329991 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-2996a967-3b7c-4104-aa56-05bdf97b60e6 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:28.339806 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 861/1709] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 13:46:27 2022] PUT /identity/v3/projects/8b3c5544eaf548beb951ab4548e6dbd0/users/fb001c7fda1b4bd492c981b7298c7239/roles/6e0201a9a6484f7d8b112b1eafff70bd => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:46:28.339806 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc32e73d-9e7e-4276-926b-dd63f6e27c00 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:28.340153 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-2996a967-3b7c-4104-aa56-05bdf97b60e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:28.506783 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 849/1710] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:46:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:46:28.510569 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2996a967-3b7c-4104-aa56-05bdf97b60e6 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:28.511323 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc32e73d-9e7e-4276-926b-dd63f6e27c00 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:28.512728 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-2996a967-3b7c-4104-aa56-05bdf97b60e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:28.522475 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b63e34-04c4-424f-8e2a-9aaa83c6964f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:28.530417 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b63e34-04c4-424f-8e2a-9aaa83c6964f admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:28.531060 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b63e34-04c4-424f-8e2a-9aaa83c6964f admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:28.531862 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b63e34-04c4-424f-8e2a-9aaa83c6964f admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:28.538231 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b63e34-04c4-424f-8e2a-9aaa83c6964f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:28.541440 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b63e34-04c4-424f-8e2a-9aaa83c6964f admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:28.562445 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 862/1711] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:46:28 2022] GET /identity/v3/roles => generated 1420 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:46:28.569416 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd753539-2f92-4342-a507-f20a4d3b4fec None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:28.572830 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd753539-2f92-4342-a507-f20a4d3b4fec None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:28.572830 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd753539-2f92-4342-a507-f20a4d3b4fec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:28.587805 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:INSERT=2,keystone:DELETE=3,keystone:SELECT=95 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:28.630885 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-fd753539-2f92-4342-a507-f20a4d3b4fec None None] MFA Rules not processed for user `fb001c7fda1b4bd492c981b7298c7239`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:28.633825 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-fd753539-2f92-4342-a507-f20a4d3b4fec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:28.655242 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2996a967-3b7c-4104-aa56-05bdf97b60e6 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:28.670331 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e14a7b-f6d4-480b-91e4-31ca169b954a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:28.672129 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e14a7b-f6d4-480b-91e4-31ca169b954a service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:28.673353 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e14a7b-f6d4-480b-91e4-31ca169b954a service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:28.675143 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e14a7b-f6d4-480b-91e4-31ca169b954a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:28.709079 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e14a7b-f6d4-480b-91e4-31ca169b954a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:28.710799 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e14a7b-f6d4-480b-91e4-31ca169b954a service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:28.749517 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 850/1712] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:46:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:46:28.753573 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e14a7b-f6d4-480b-91e4-31ca169b954a service neutron] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:28.769112 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e14a7b-f6d4-480b-91e4-31ca169b954a service neutron] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:28.783978 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec7cb8a-0043-4ff4-a625-2fb112edaed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:28.790424 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec7cb8a-0043-4ff4-a625-2fb112edaed5 admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:28.791586 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec7cb8a-0043-4ff4-a625-2fb112edaed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:28.791967 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec7cb8a-0043-4ff4-a625-2fb112edaed5 admin admin] PATH_INFO: `/v3/projects/63c5b644c2d646478c672a5d7cf6bcc2/users/85f4eb91a7e843e284628d423cf22f7a/roles/6e0201a9a6484f7d8b112b1eafff70bd` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:28.798979 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec7cb8a-0043-4ff4-a625-2fb112edaed5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63c5b644c2d646478c672a5d7cf6bcc2, user_id=85f4eb91a7e843e284628d423cf22f7a, role_id=6e0201a9a6484f7d8b112b1eafff70bd)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:28.804998 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec7cb8a-0043-4ff4-a625-2fb112edaed5 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:28.840669 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 863/1713] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:46:28 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:46:28.845227 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd753539-2f92-4342-a507-f20a4d3b4fec None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:28.845320 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 851/1714] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 13:46:28 2022] PUT /identity/v3/projects/63c5b644c2d646478c672a5d7cf6bcc2/users/85f4eb91a7e843e284628d423cf22f7a/roles/6e0201a9a6484f7d8b112b1eafff70bd => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:46:28.859246 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec7cb8a-0043-4ff4-a625-2fb112edaed5 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:28.861625 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-3ec7cb8a-0043-4ff4-a625-2fb112edaed5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:28.869694 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd753539-2f92-4342-a507-f20a4d3b4fec None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:28.976952 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b26a07-8d39-43b8-9414-d918c45e8fd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:28.977641 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b26a07-8d39-43b8-9414-d918c45e8fd3 service neutron] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:28.977921 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b26a07-8d39-43b8-9414-d918c45e8fd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:28.978203 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b26a07-8d39-43b8-9414-d918c45e8fd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:28.995243 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec7cb8a-0043-4ff4-a625-2fb112edaed5 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:29.006639 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f653207a-2657-4e39-9c0d-fe9eb06fbf32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:29.007458 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f653207a-2657-4e39-9c0d-fe9eb06fbf32 service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:29.007878 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f653207a-2657-4e39-9c0d-fe9eb06fbf32 service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:29.008374 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f653207a-2657-4e39-9c0d-fe9eb06fbf32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:29.076523 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:29.120560 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b26a07-8d39-43b8-9414-d918c45e8fd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:29.122200 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b26a07-8d39-43b8-9414-d918c45e8fd3 service neutron] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:29.144212 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 864/1715] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:46:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:46:29.147805 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b779921-589a-4798-b909-f0393fc2aaad None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:29.148435 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b779921-589a-4798-b909-f0393fc2aaad None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:29.148860 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b779921-589a-4798-b909-f0393fc2aaad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:29.211886 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-3b779921-589a-4798-b909-f0393fc2aaad None None] MFA Rules not processed for user `1f5b2d80af624d45977460439d4eec2b`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:29.213234 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-3b779921-589a-4798-b909-f0393fc2aaad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:29.252909 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f653207a-2657-4e39-9c0d-fe9eb06fbf32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:29.258560 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f653207a-2657-4e39-9c0d-fe9eb06fbf32 service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:29.275130 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:29.306543 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 852/1716] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:46:28 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 13:46:29.369817 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 865/1717] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:46:29 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:46:29.382705 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c061d34-788b-4b87-b7b3-dfdcc3ac8d70 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:29.383278 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c061d34-788b-4b87-b7b3-dfdcc3ac8d70 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:29.384037 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c061d34-788b-4b87-b7b3-dfdcc3ac8d70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:29.449809 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-9c061d34-788b-4b87-b7b3-dfdcc3ac8d70 None None] MFA Rules not processed for user `1f5b2d80af624d45977460439d4eec2b`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:29.453874 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-9c061d34-788b-4b87-b7b3-dfdcc3ac8d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:29.478057 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 853/1718] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:46:29 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:46:29.483513 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b779921-589a-4798-b909-f0393fc2aaad None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:29.485008 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-3b779921-589a-4798-b909-f0393fc2aaad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:29.490015 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c061d34-788b-4b87-b7b3-dfdcc3ac8d70 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:29.525591 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c061d34-788b-4b87-b7b3-dfdcc3ac8d70 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:29.538872 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d58508-d81f-4415-9fab-8d045bf0eef3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:29.539910 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d58508-d81f-4415-9fab-8d045bf0eef3 service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:29.543400 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d58508-d81f-4415-9fab-8d045bf0eef3 service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:29.543977 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d58508-d81f-4415-9fab-8d045bf0eef3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:29.549085 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-d4d58508-d81f-4415-9fab-8d045bf0eef3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:29.658151 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b779921-589a-4798-b909-f0393fc2aaad None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:29.667578 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d7dc46-f41b-4d1c-9a16-e99a6d8ec12d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c69161cd520b4b29bf50f3bf2f1c033b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:29.668719 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d7dc46-f41b-4d1c-9a16-e99a6d8ec12d service nova] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:29.669226 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d7dc46-f41b-4d1c-9a16-e99a6d8ec12d service nova] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:29.670994 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d7dc46-f41b-4d1c-9a16-e99a6d8ec12d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:29.687809 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d7dc46-f41b-4d1c-9a16-e99a6d8ec12d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:29.690791 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d7dc46-f41b-4d1c-9a16-e99a6d8ec12d service nova] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:29.717089 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 866/1719] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:46:29 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:46:29.732686 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d58508-d81f-4415-9fab-8d045bf0eef3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:29.734632 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d58508-d81f-4415-9fab-8d045bf0eef3 service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:29.764364 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 854/1720] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:46:29 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:46:30.428615 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d7dc46-f41b-4d1c-9a16-e99a6d8ec12d service nova] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:30.431332 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-50d7dc46-f41b-4d1c-9a16-e99a6d8ec12d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:30.471291 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d7dc46-f41b-4d1c-9a16-e99a6d8ec12d service nova] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:30.499476 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd5af85-26ab-4913-a4d3-0cb466b2c13a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:30.500167 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd5af85-26ab-4913-a4d3-0cb466b2c13a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:30.500459 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd5af85-26ab-4913-a4d3-0cb466b2c13a admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:30.503225 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd5af85-26ab-4913-a4d3-0cb466b2c13a admin admin] PATH_INFO: `/v3/projects/73efb3b87fed4f1c9ce8b5eca4d2dd0f` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:30.505293 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd5af85-26ab-4913-a4d3-0cb466b2c13a admin admin] RBAC: Authorizing `identity:delete_project(project_id=73efb3b87fed4f1c9ce8b5eca4d2dd0f)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:30.509511 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd5af85-26ab-4913-a4d3-0cb466b2c13a admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:30.640169 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a8779f-16c2-4f4f-8cb3-c26e8f33f409 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:30.640685 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a8779f-16c2-4f4f-8cb3-c26e8f33f409 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:30.640760 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a8779f-16c2-4f4f-8cb3-c26e8f33f409 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:30.644746 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 855/1721] 10.222.0.58 () {54 vars in 912 bytes} [Tue Jul 26 13:46:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:46:30.646649 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5909976-4a03-4f2f-a4aa-316cd3aa6df1 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:30.647073 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5909976-4a03-4f2f-a4aa-316cd3aa6df1 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:30.647464 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5909976-4a03-4f2f-a4aa-316cd3aa6df1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:30.651748 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-efd5af85-26ab-4913-a4d3-0cb466b2c13a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73efb3b87fed4f1c9ce8b5eca4d2dd0f', 'request_id': 'req-efd5af85-26ab-4913-a4d3-0cb466b2c13a'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:30.674745 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-efd5af85-26ab-4913-a4d3-0cb466b2c13a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73efb3b87fed4f1c9ce8b5eca4d2dd0f', 'request_id': 'req-efd5af85-26ab-4913-a4d3-0cb466b2c13a'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:30.685893 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-efd5af85-26ab-4913-a4d3-0cb466b2c13a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73efb3b87fed4f1c9ce8b5eca4d2dd0f', 'request_id': 'req-efd5af85-26ab-4913-a4d3-0cb466b2c13a'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:30.712204 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-efd5af85-26ab-4913-a4d3-0cb466b2c13a admin admin] The token cache is being invalidate because project 73efb3b87fed4f1c9ce8b5eca4d2dd0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86933) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 13:46:30.712820 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-efd5af85-26ab-4913-a4d3-0cb466b2c13a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:30.716223 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-efd5af85-26ab-4913-a4d3-0cb466b2c13a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:30.721710 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 867/1722] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 13:46:30 2022] DELETE /identity/v3/projects/73efb3b87fed4f1c9ce8b5eca4d2dd0f => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:46:30.724184 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-e5909976-4a03-4f2f-a4aa-316cd3aa6df1 None None] MFA Rules not processed for user `1747d40a285942eb9b4d5bc14136d88a`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:30.724660 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7957a380-a06e-4952-b3a4-2fe52bc6d7b0 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:30.726352 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-e5909976-4a03-4f2f-a4aa-316cd3aa6df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:30.726945 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7957a380-a06e-4952-b3a4-2fe52bc6d7b0 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:30.727368 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7957a380-a06e-4952-b3a4-2fe52bc6d7b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:30.840239 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-7957a380-a06e-4952-b3a4-2fe52bc6d7b0 None None] MFA Rules not processed for user `3223a78866d94c2ba66402911c2dfae9`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:30.845352 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-7957a380-a06e-4952-b3a4-2fe52bc6d7b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:30.894888 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:30.945404 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 856/1723] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:46:30 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:46:30.946649 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5909976-4a03-4f2f-a4aa-316cd3aa6df1 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:30.949933 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-e5909976-4a03-4f2f-a4aa-316cd3aa6df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:31.056514 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 868/1724] 10.222.0.58 () {58 vars in 1009 bytes} [Tue Jul 26 13:46:30 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:46:31.057751 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7957a380-a06e-4952-b3a4-2fe52bc6d7b0 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:31.061868 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-7957a380-a06e-4952-b3a4-2fe52bc6d7b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:31.085466 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:31.178840 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5909976-4a03-4f2f-a4aa-316cd3aa6df1 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:31.193920 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bca382-cf66-46b5-84d5-5a4c74f81fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:31.194593 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bca382-cf66-46b5-84d5-5a4c74f81fd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:31.195219 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bca382-cf66-46b5-84d5-5a4c74f81fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:31.195219 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bca382-cf66-46b5-84d5-5a4c74f81fd0 admin admin] PATH_INFO: `/v3/projects/f833fde4e16046029aff7fb83f68b466` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:31.201706 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bca382-cf66-46b5-84d5-5a4c74f81fd0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f833fde4e16046029aff7fb83f68b466)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:31.206057 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bca382-cf66-46b5-84d5-5a4c74f81fd0 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:31.250404 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7957a380-a06e-4952-b3a4-2fe52bc6d7b0 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:31.263036 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f9a90c-195f-4603-8a46-bf4c485af068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:31.263719 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f9a90c-195f-4603-8a46-bf4c485af068 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:31.264010 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f9a90c-195f-4603-8a46-bf4c485af068 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:31.264292 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f9a90c-195f-4603-8a46-bf4c485af068 admin admin] PATH_INFO: `/v3/users/bc27cf21a70941559d3f89ca36cae5e1` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:31.267878 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f9a90c-195f-4603-8a46-bf4c485af068 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc27cf21a70941559d3f89ca36cae5e1)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:31.270344 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f9a90c-195f-4603-8a46-bf4c485af068 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:31.316743 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-79bca382-cf66-46b5-84d5-5a4c74f81fd0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f833fde4e16046029aff7fb83f68b466', 'request_id': 'req-79bca382-cf66-46b5-84d5-5a4c74f81fd0'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:31.331012 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-79bca382-cf66-46b5-84d5-5a4c74f81fd0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f833fde4e16046029aff7fb83f68b466', 'request_id': 'req-79bca382-cf66-46b5-84d5-5a4c74f81fd0'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:31.339880 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-79bca382-cf66-46b5-84d5-5a4c74f81fd0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f833fde4e16046029aff7fb83f68b466', 'request_id': 'req-79bca382-cf66-46b5-84d5-5a4c74f81fd0'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:31.375369 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-79bca382-cf66-46b5-84d5-5a4c74f81fd0 admin admin] The token cache is being invalidate because project f833fde4e16046029aff7fb83f68b466 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 13:46:31.375786 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-79bca382-cf66-46b5-84d5-5a4c74f81fd0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:31.377514 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-00f9a90c-195f-4603-8a46-bf4c485af068 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc27cf21a70941559d3f89ca36cae5e1'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:31.380566 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-79bca382-cf66-46b5-84d5-5a4c74f81fd0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:31.386642 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 857/1725] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 13:46:30 2022] DELETE /identity/v3/projects/f833fde4e16046029aff7fb83f68b466 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:46:31.387618 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bca382-cf66-46b5-84d5-5a4c74f81fd0 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:31.393108 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-79bca382-cf66-46b5-84d5-5a4c74f81fd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:31.395938 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-00f9a90c-195f-4603-8a46-bf4c485af068 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc27cf21a70941559d3f89ca36cae5e1'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:31.409618 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-00f9a90c-195f-4603-8a46-bf4c485af068 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc27cf21a70941559d3f89ca36cae5e1'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:31.411456 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-00f9a90c-195f-4603-8a46-bf4c485af068 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc27cf21a70941559d3f89ca36cae5e1'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:31.413773 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-00f9a90c-195f-4603-8a46-bf4c485af068 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc27cf21a70941559d3f89ca36cae5e1'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:31.464234 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 869/1726] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 13:46:31 2022] DELETE /identity/v3/users/bc27cf21a70941559d3f89ca36cae5e1 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:46:31.464234 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f81677-2d73-40c5-9a02-cf2d6fd6d82e None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:31.464234 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f81677-2d73-40c5-9a02-cf2d6fd6d82e None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:31.465088 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f81677-2d73-40c5-9a02-cf2d6fd6d82e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:31.549513 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-f0f81677-2d73-40c5-9a02-cf2d6fd6d82e None None] MFA Rules not processed for user `85f4eb91a7e843e284628d423cf22f7a`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:31.550886 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-f0f81677-2d73-40c5-9a02-cf2d6fd6d82e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:31.596603 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bca382-cf66-46b5-84d5-5a4c74f81fd0 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:31.658812 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:31.710635 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 870/1727] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:46:31 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:46:31.711849 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f81677-2d73-40c5-9a02-cf2d6fd6d82e None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:31.717708 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-f0f81677-2d73-40c5-9a02-cf2d6fd6d82e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:31.833785 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:31.881718 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7f99e1-f58c-4c0f-ab71-9ed9482a727a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c69161cd520b4b29bf50f3bf2f1c033b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:31.882293 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7f99e1-f58c-4c0f-ab71-9ed9482a727a service nova] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:31.882573 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7f99e1-f58c-4c0f-ab71-9ed9482a727a service nova] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:31.882848 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7f99e1-f58c-4c0f-ab71-9ed9482a727a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:31.895921 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-4d7f99e1-f58c-4c0f-ab71-9ed9482a727a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:31.918360 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f81677-2d73-40c5-9a02-cf2d6fd6d82e None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:31.931606 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ade4920-d66d-488a-8021-1da8a7cb3cc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:31.932780 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ade4920-d66d-488a-8021-1da8a7cb3cc2 service neutron] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:31.933413 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ade4920-d66d-488a-8021-1da8a7cb3cc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:31.933952 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ade4920-d66d-488a-8021-1da8a7cb3cc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:31.936927 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-4ade4920-d66d-488a-8021-1da8a7cb3cc2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:31.950545 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ade4920-d66d-488a-8021-1da8a7cb3cc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:31.952129 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ade4920-d66d-488a-8021-1da8a7cb3cc2 service neutron] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:31.979590 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 871/1728] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:46:31 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:46:31.982406 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bdc998-5a9c-49d7-9d28-c3b83b249841 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:31.983762 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bdc998-5a9c-49d7-9d28-c3b83b249841 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:31.984160 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bdc998-5a9c-49d7-9d28-c3b83b249841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:32.050012 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7f99e1-f58c-4c0f-ab71-9ed9482a727a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:32.051620 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7f99e1-f58c-4c0f-ab71-9ed9482a727a service nova] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:32.077364 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-24bdc998-5a9c-49d7-9d28-c3b83b249841 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:32.081547 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-24bdc998-5a9c-49d7-9d28-c3b83b249841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:32.083279 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 858/1729] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:46:31 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 696 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:46:32.084451 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60181860-ac0f-41f9-85d0-50a1a3f53c02 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:32.084665 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60181860-ac0f-41f9-85d0-50a1a3f53c02 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:32.084941 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60181860-ac0f-41f9-85d0-50a1a3f53c02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:32.184620 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-60181860-ac0f-41f9-85d0-50a1a3f53c02 None None] MFA Rules not processed for user `56db6603f359477885b5151dec71c096`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:32.190381 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-60181860-ac0f-41f9-85d0-50a1a3f53c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:32.286566 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 872/1730] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:46:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:46:32.296043 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eece21f-3068-49b4-9f05-c425e66b1a74 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:32.296043 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eece21f-3068-49b4-9f05-c425e66b1a74 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:32.296043 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eece21f-3068-49b4-9f05-c425e66b1a74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:32.389765 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-5eece21f-3068-49b4-9f05-c425e66b1a74 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:32.397360 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-5eece21f-3068-49b4-9f05-c425e66b1a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:32.436268 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 873/1731] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:46:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 13:46:32.441306 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eece21f-3068-49b4-9f05-c425e66b1a74 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:32.467464 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 859/1732] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:46:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 383 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:46:32.471475 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eece21f-3068-49b4-9f05-c425e66b1a74 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:32.483754 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60181860-ac0f-41f9-85d0-50a1a3f53c02 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:32.486809 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-60181860-ac0f-41f9-85d0-50a1a3f53c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:32.494441 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca2cdb3-868c-4438-871d-92be00486877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:32.495231 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca2cdb3-868c-4438-871d-92be00486877 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:32.495513 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca2cdb3-868c-4438-871d-92be00486877 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:32.496364 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca2cdb3-868c-4438-871d-92be00486877 admin admin] PATH_INFO: `/v3/domains` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:32.498389 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca2cdb3-868c-4438-871d-92be00486877 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:46:32.498699 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca2cdb3-868c-4438-871d-92be00486877 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:32.507690 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca2cdb3-868c-4438-871d-92be00486877 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:32.540256 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60181860-ac0f-41f9-85d0-50a1a3f53c02 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:32.554365 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 874/1733] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Jul 26 13:46:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:46:32.555302 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca2cdb3-868c-4438-871d-92be00486877 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:32.564533 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd96a77d-8fab-49cf-a548-777c7b17ddd0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c69161cd520b4b29bf50f3bf2f1c033b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:32.565190 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd96a77d-8fab-49cf-a548-777c7b17ddd0 service nova] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:32.565503 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd96a77d-8fab-49cf-a548-777c7b17ddd0 service nova] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:32.565822 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd96a77d-8fab-49cf-a548-777c7b17ddd0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:32.584404 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd96a77d-8fab-49cf-a548-777c7b17ddd0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:32.586492 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd96a77d-8fab-49cf-a548-777c7b17ddd0 service nova] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:32.594664 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca2cdb3-868c-4438-871d-92be00486877 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:32.609841 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 860/1734] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:46:32 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 13:46:32.619143 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23947033-5c45-4a05-a0c3-b60db4da0656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:32.620164 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23947033-5c45-4a05-a0c3-b60db4da0656 admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:32.620520 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23947033-5c45-4a05-a0c3-b60db4da0656 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:32.620848 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23947033-5c45-4a05-a0c3-b60db4da0656 admin admin] PATH_INFO: `/v3/projects` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:32.621573 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23947033-5c45-4a05-a0c3-b60db4da0656 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:32.623468 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23947033-5c45-4a05-a0c3-b60db4da0656 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:32.749981 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 875/1735] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 13:46:32 2022] POST /identity/v3/projects => generated 369 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:46:32.752329 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd96a77d-8fab-49cf-a548-777c7b17ddd0 service nova] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:32.773222 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd96a77d-8fab-49cf-a548-777c7b17ddd0 service nova] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:33.032104 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83daa8ff-177b-4b4d-b4c3-daa134071119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:33.032878 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83daa8ff-177b-4b4d-b4c3-daa134071119 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:33.033171 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83daa8ff-177b-4b4d-b4c3-daa134071119 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:33.033453 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83daa8ff-177b-4b4d-b4c3-daa134071119 admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:33.034174 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83daa8ff-177b-4b4d-b4c3-daa134071119 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:33.036296 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83daa8ff-177b-4b4d-b4c3-daa134071119 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:33.059324 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23947033-5c45-4a05-a0c3-b60db4da0656 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:33.061299 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-23947033-5c45-4a05-a0c3-b60db4da0656 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:33.090305 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23947033-5c45-4a05-a0c3-b60db4da0656 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:33.097419 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 861/1736] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 13:46:32 2022] POST /identity/v3/users => generated 285 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:46:33.101115 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83daa8ff-177b-4b4d-b4c3-daa134071119 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:33.103475 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d1b62d-4204-4897-bafa-5043b6b6cdad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:33.104150 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d1b62d-4204-4897-bafa-5043b6b6cdad admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:33.104434 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d1b62d-4204-4897-bafa-5043b6b6cdad admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:33.104707 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d1b62d-4204-4897-bafa-5043b6b6cdad admin admin] PATH_INFO: `/v3/projects/52de86fb9f0d4b67afa32693f2ae242f` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:33.108277 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d1b62d-4204-4897-bafa-5043b6b6cdad admin admin] RBAC: Authorizing `identity:delete_project(project_id=52de86fb9f0d4b67afa32693f2ae242f)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:33.113913 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d1b62d-4204-4897-bafa-5043b6b6cdad admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:33.120565 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:33.137400 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83daa8ff-177b-4b4d-b4c3-daa134071119 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:33.169270 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d950fa8-bae0-4a49-8e34-eceb73009a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:33.169970 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d950fa8-bae0-4a49-8e34-eceb73009a09 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:33.170259 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d950fa8-bae0-4a49-8e34-eceb73009a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:33.170541 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d950fa8-bae0-4a49-8e34-eceb73009a09 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:33.171266 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d950fa8-bae0-4a49-8e34-eceb73009a09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:33.173061 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d950fa8-bae0-4a49-8e34-eceb73009a09 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:33.214648 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 862/1737] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:46:33 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:46:33.218445 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d950fa8-bae0-4a49-8e34-eceb73009a09 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:33.233343 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:33.243858 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d950fa8-bae0-4a49-8e34-eceb73009a09 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:33.282123 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-c8d1b62d-4204-4897-bafa-5043b6b6cdad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52de86fb9f0d4b67afa32693f2ae242f', 'request_id': 'req-c8d1b62d-4204-4897-bafa-5043b6b6cdad'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:33.301233 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-c8d1b62d-4204-4897-bafa-5043b6b6cdad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52de86fb9f0d4b67afa32693f2ae242f', 'request_id': 'req-c8d1b62d-4204-4897-bafa-5043b6b6cdad'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:33.308894 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-c8d1b62d-4204-4897-bafa-5043b6b6cdad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52de86fb9f0d4b67afa32693f2ae242f', 'request_id': 'req-c8d1b62d-4204-4897-bafa-5043b6b6cdad'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:33.332174 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-c8d1b62d-4204-4897-bafa-5043b6b6cdad admin admin] The token cache is being invalidate because project 52de86fb9f0d4b67afa32693f2ae242f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86933) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 13:46:33.332471 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-c8d1b62d-4204-4897-bafa-5043b6b6cdad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:33.337044 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-c8d1b62d-4204-4897-bafa-5043b6b6cdad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:33.341046 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 876/1738] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 13:46:33 2022] DELETE /identity/v3/projects/52de86fb9f0d4b67afa32693f2ae242f => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 13:46:33.347588 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb71e21b-4417-42a7-9035-4c11116b1060 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:33.350675 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb71e21b-4417-42a7-9035-4c11116b1060 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:33.351092 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb71e21b-4417-42a7-9035-4c11116b1060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:33.421455 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-eb71e21b-4417-42a7-9035-4c11116b1060 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:33.422585 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-eb71e21b-4417-42a7-9035-4c11116b1060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:33.441091 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0febfd28-2c12-4f92-902a-662537e571fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:33.442263 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0febfd28-2c12-4f92-902a-662537e571fa admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:33.442741 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0febfd28-2c12-4f92-902a-662537e571fa admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:33.444346 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0febfd28-2c12-4f92-902a-662537e571fa admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:33.445270 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0febfd28-2c12-4f92-902a-662537e571fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:33.447380 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0febfd28-2c12-4f92-902a-662537e571fa admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:33.461720 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 877/1739] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:46:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:46:33.474004 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 863/1740] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:46:33 2022] GET /identity/v3/roles => generated 1420 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:46:33.474943 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f076a4d-b781-4735-aa86-ae11a754a0ce None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:33.475233 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f076a4d-b781-4735-aa86-ae11a754a0ce None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:33.475518 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f076a4d-b781-4735-aa86-ae11a754a0ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:33.485534 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb71e21b-4417-42a7-9035-4c11116b1060 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:33.491295 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-eb71e21b-4417-42a7-9035-4c11116b1060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:33.505031 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-7f076a4d-b781-4735-aa86-ae11a754a0ce None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:33.511244 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-7f076a4d-b781-4735-aa86-ae11a754a0ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:33.537265 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb71e21b-4417-42a7-9035-4c11116b1060 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:33.549408 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 864/1741] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:46:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:46:33.552005 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f076a4d-b781-4735-aa86-ae11a754a0ce None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:33.552753 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e30b1a-c63f-42b6-858c-799bbb973d99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:33.553877 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e30b1a-c63f-42b6-858c-799bbb973d99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:33.554879 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-7f076a4d-b781-4735-aa86-ae11a754a0ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:33.556701 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e30b1a-c63f-42b6-858c-799bbb973d99 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:33.557061 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e30b1a-c63f-42b6-858c-799bbb973d99 admin admin] PATH_INFO: `/v3/users/b48f5ad3693743c6b2a1ad330f059d81` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:33.559392 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e30b1a-c63f-42b6-858c-799bbb973d99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b48f5ad3693743c6b2a1ad330f059d81)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:33.563056 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e30b1a-c63f-42b6-858c-799bbb973d99 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:33.595173 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f076a4d-b781-4735-aa86-ae11a754a0ce None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:33.623846 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18361db-713f-4c87-aff0-b2ed4357140d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:33.625210 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18361db-713f-4c87-aff0-b2ed4357140d admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:33.625532 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18361db-713f-4c87-aff0-b2ed4357140d admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:33.626367 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18361db-713f-4c87-aff0-b2ed4357140d admin admin] PATH_INFO: `/v3/projects/8071f81ccad0427db5883efba1c28b0b/users/d24e49566a4248c1ba8d8db011bdd603/roles/759f389b221049c4b01faf3b984cd8d7` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:33.674666 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18361db-713f-4c87-aff0-b2ed4357140d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8071f81ccad0427db5883efba1c28b0b, user_id=d24e49566a4248c1ba8d8db011bdd603, role_id=759f389b221049c4b01faf3b984cd8d7)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:33.677735 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18361db-713f-4c87-aff0-b2ed4357140d admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:33.705827 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 865/1742] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 13:46:33 2022] PUT /identity/v3/projects/8071f81ccad0427db5883efba1c28b0b/users/d24e49566a4248c1ba8d8db011bdd603/roles/759f389b221049c4b01faf3b984cd8d7 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:46:33.718676 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-a7e30b1a-c63f-42b6-858c-799bbb973d99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b48f5ad3693743c6b2a1ad330f059d81'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:33.720313 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18361db-713f-4c87-aff0-b2ed4357140d admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:33.734632 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-a7e30b1a-c63f-42b6-858c-799bbb973d99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b48f5ad3693743c6b2a1ad330f059d81'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:33.747201 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-a7e30b1a-c63f-42b6-858c-799bbb973d99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b48f5ad3693743c6b2a1ad330f059d81'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:33.748291 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-a7e30b1a-c63f-42b6-858c-799bbb973d99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b48f5ad3693743c6b2a1ad330f059d81'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:33.749199 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-a7e30b1a-c63f-42b6-858c-799bbb973d99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b48f5ad3693743c6b2a1ad330f059d81'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:33.810876 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 878/1743] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 13:46:33 2022] DELETE /identity/v3/users/b48f5ad3693743c6b2a1ad330f059d81 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:46:33.812395 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61850b7-81ec-43f6-ac05-6886069d27cc None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:33.812684 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61850b7-81ec-43f6-ac05-6886069d27cc None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:33.812956 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61850b7-81ec-43f6-ac05-6886069d27cc None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:33.827542 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 879/1744] 10.222.0.58 () {54 vars in 961 bytes} [Tue Jul 26 13:46:33 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:46:33.832952 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b612ea6-006e-4cf9-ad9a-662b8fb8c073 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:33.833279 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b612ea6-006e-4cf9-ad9a-662b8fb8c073 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:33.833605 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b612ea6-006e-4cf9-ad9a-662b8fb8c073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:33.894780 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18361db-713f-4c87-aff0-b2ed4357140d admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:33.905414 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-0b612ea6-006e-4cf9-ad9a-662b8fb8c073 None None] MFA Rules not processed for user `d24e49566a4248c1ba8d8db011bdd603`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:33.907040 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-0b612ea6-006e-4cf9-ad9a-662b8fb8c073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:34.019598 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 880/1745] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:46:33 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:46:34.023390 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099d2f48-77f3-45bb-bf5d-746f847b8f27 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:34.027545 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099d2f48-77f3-45bb-bf5d-746f847b8f27 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:34.027887 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099d2f48-77f3-45bb-bf5d-746f847b8f27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:34.032674 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d674e3-b5b4-46a5-bf94-c8ab5f607098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:34.033340 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d674e3-b5b4-46a5-bf94-c8ab5f607098 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:34.033663 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d674e3-b5b4-46a5-bf94-c8ab5f607098 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:34.033989 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d674e3-b5b4-46a5-bf94-c8ab5f607098 admin admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:34.034712 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d674e3-b5b4-46a5-bf94-c8ab5f607098 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:46:34.035025 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d674e3-b5b4-46a5-bf94-c8ab5f607098 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:34.038874 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d674e3-b5b4-46a5-bf94-c8ab5f607098 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:34.053096 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:34.067611 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 866/1746] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Jul 26 13:46:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:46:34.073712 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a40654-717b-43b8-9b94-4e280e4ef492 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:34.076379 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a40654-717b-43b8-9b94-4e280e4ef492 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:34.076712 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a40654-717b-43b8-9b94-4e280e4ef492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:34.090546 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-099d2f48-77f3-45bb-bf5d-746f847b8f27 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:34.091770 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-099d2f48-77f3-45bb-bf5d-746f847b8f27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:34.111986 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-24a40654-717b-43b8-9b94-4e280e4ef492 None None] MFA Rules not processed for user `835cbc13756b41b598142df3e35f5492`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:34.114014 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-24a40654-717b-43b8-9b94-4e280e4ef492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:34.118696 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 881/1747] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:46:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:46:34.124057 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099d2f48-77f3-45bb-bf5d-746f847b8f27 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:34.128499 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-099d2f48-77f3-45bb-bf5d-746f847b8f27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:34.155037 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099d2f48-77f3-45bb-bf5d-746f847b8f27 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:34.171578 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63944119-71b8-4065-8cb9-a2a09f431ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:34.172226 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63944119-71b8-4065-8cb9-a2a09f431ac5 admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:34.172504 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63944119-71b8-4065-8cb9-a2a09f431ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:34.173316 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63944119-71b8-4065-8cb9-a2a09f431ac5 admin admin] PATH_INFO: `/v3/projects` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:34.174036 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63944119-71b8-4065-8cb9-a2a09f431ac5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:34.175969 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63944119-71b8-4065-8cb9-a2a09f431ac5 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:34.193546 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:34.252535 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 882/1748] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 13:46:34 2022] POST /identity/v3/projects => generated 373 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:46:34.253400 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63944119-71b8-4065-8cb9-a2a09f431ac5 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:34.257593 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-63944119-71b8-4065-8cb9-a2a09f431ac5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:34.267843 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 867/1749] 10.222.0.58 () {58 vars in 1058 bytes} [Tue Jul 26 13:46:34 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:46:34.276429 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a40654-717b-43b8-9b94-4e280e4ef492 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:34.277921 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-24a40654-717b-43b8-9b94-4e280e4ef492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:34.386518 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63944119-71b8-4065-8cb9-a2a09f431ac5 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:34.393945 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf80faf-17d1-4c40-980c-2c923a61363e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:34.395927 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf80faf-17d1-4c40-980c-2c923a61363e admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:34.396498 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf80faf-17d1-4c40-980c-2c923a61363e admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:34.396964 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf80faf-17d1-4c40-980c-2c923a61363e admin admin] PATH_INFO: `/v3/projects` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:34.397942 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf80faf-17d1-4c40-980c-2c923a61363e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:34.400392 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf80faf-17d1-4c40-980c-2c923a61363e admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:34.415935 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a40654-717b-43b8-9b94-4e280e4ef492 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:34.457154 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 883/1750] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 13:46:34 2022] POST /identity/v3/projects => generated 365 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:46:34.461351 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf80faf-17d1-4c40-980c-2c923a61363e admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:34.509873 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c127b1d-2af5-4d16-b740-eacd621ddf13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:34.510728 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c127b1d-2af5-4d16-b740-eacd621ddf13 service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:34.511209 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c127b1d-2af5-4d16-b740-eacd621ddf13 service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:34.511643 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c127b1d-2af5-4d16-b740-eacd621ddf13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:34.533185 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c127b1d-2af5-4d16-b740-eacd621ddf13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:34.536932 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c127b1d-2af5-4d16-b740-eacd621ddf13 service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:34.564538 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 868/1751] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:46:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:46:34.565997 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c127b1d-2af5-4d16-b740-eacd621ddf13 service neutron] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:34.601591 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf80faf-17d1-4c40-980c-2c923a61363e admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:34.607414 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fe29eb-751e-46bf-8202-4d93b99d8cb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:34.608267 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fe29eb-751e-46bf-8202-4d93b99d8cb3 admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:34.608608 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fe29eb-751e-46bf-8202-4d93b99d8cb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:34.609059 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fe29eb-751e-46bf-8202-4d93b99d8cb3 admin admin] PATH_INFO: `/v3/users` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:34.609982 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fe29eb-751e-46bf-8202-4d93b99d8cb3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:34.613138 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fe29eb-751e-46bf-8202-4d93b99d8cb3 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:34.667828 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:34.678629 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 884/1752] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 13:46:34 2022] POST /identity/v3/users => generated 293 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:46:34.685270 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c127b1d-2af5-4d16-b740-eacd621ddf13 service neutron] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:34.685696 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fe29eb-751e-46bf-8202-4d93b99d8cb3 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:34.736639 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fe29eb-751e-46bf-8202-4d93b99d8cb3 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:34.750860 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161fd3bc-e486-4cd8-beeb-13e17dd595f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:34.752477 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161fd3bc-e486-4cd8-beeb-13e17dd595f5 admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:34.752768 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161fd3bc-e486-4cd8-beeb-13e17dd595f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:34.753047 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161fd3bc-e486-4cd8-beeb-13e17dd595f5 admin admin] PATH_INFO: `/v3/users` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:34.756293 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161fd3bc-e486-4cd8-beeb-13e17dd595f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:34.761086 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161fd3bc-e486-4cd8-beeb-13e17dd595f5 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:34.789014 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03cc2ea-e9be-4c04-9169-4849f296346f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '835cbc13756b41b598142df3e35f5492', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:34.789684 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03cc2ea-e9be-4c04-9169-4849f296346f service placement] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:34.789978 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03cc2ea-e9be-4c04-9169-4849f296346f service placement] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:34.790242 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03cc2ea-e9be-4c04-9169-4849f296346f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:34.792208 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-c03cc2ea-e9be-4c04-9169-4849f296346f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:34.809288 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03cc2ea-e9be-4c04-9169-4849f296346f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:34.810909 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03cc2ea-e9be-4c04-9169-4849f296346f service placement] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:34.815551 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 885/1753] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 13:46:34 2022] POST /identity/v3/users => generated 283 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:46:34.823665 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7931e9-dfc0-4440-b8f1-d568ea6104d4 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:34.824090 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7931e9-dfc0-4440-b8f1-d568ea6104d4 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:34.826577 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7931e9-dfc0-4440-b8f1-d568ea6104d4 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:34.828118 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 886/1754] 10.222.0.58 () {54 vars in 921 bytes} [Tue Jul 26 13:46:34 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:46:34.833033 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7931e9-dfc0-4440-b8f1-d568ea6104d4 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:34.841863 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 869/1755] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:46:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:46:34.849602 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03cc2ea-e9be-4c04-9169-4849f296346f service placement] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:34.886144 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03cc2ea-e9be-4c04-9169-4849f296346f service placement] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:34.891270 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7931e9-dfc0-4440-b8f1-d568ea6104d4 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:34.906752 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ef44ca-d060-4c40-b9e3-dc70bdb3646b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:34.908108 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ef44ca-d060-4c40-b9e3-dc70bdb3646b admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:34.908395 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ef44ca-d060-4c40-b9e3-dc70bdb3646b admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:34.908681 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ef44ca-d060-4c40-b9e3-dc70bdb3646b admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:34.909383 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ef44ca-d060-4c40-b9e3-dc70bdb3646b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:34.911058 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ef44ca-d060-4c40-b9e3-dc70bdb3646b admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:34.912629 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8677b94-43cf-43f0-82ce-74d85d367dbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:34.913280 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8677b94-43cf-43f0-82ce-74d85d367dbb admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:34.917280 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8677b94-43cf-43f0-82ce-74d85d367dbb admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:34.917468 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8677b94-43cf-43f0-82ce-74d85d367dbb admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:34.918160 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8677b94-43cf-43f0-82ce-74d85d367dbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:34.923285 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8677b94-43cf-43f0-82ce-74d85d367dbb admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:34.946917 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 870/1756] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:46:34 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:46:34.948424 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf9c2df-0cb7-4ed9-87a3-9687bd8c629a None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:34.948712 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf9c2df-0cb7-4ed9-87a3-9687bd8c629a None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:34.948994 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf9c2df-0cb7-4ed9-87a3-9687bd8c629a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:34.955479 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 887/1757] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:46:34 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:46:34.956389 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8677b94-43cf-43f0-82ce-74d85d367dbb admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:34.981074 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8677b94-43cf-43f0-82ce-74d85d367dbb admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:35.000947 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caee455-44ee-4e75-9f54-77b13cd1b635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:35.001795 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caee455-44ee-4e75-9f54-77b13cd1b635 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:35.002189 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caee455-44ee-4e75-9f54-77b13cd1b635 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:35.002594 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caee455-44ee-4e75-9f54-77b13cd1b635 admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:35.003514 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caee455-44ee-4e75-9f54-77b13cd1b635 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:35.005241 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caee455-44ee-4e75-9f54-77b13cd1b635 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:35.005426 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-0cf9c2df-0cb7-4ed9-87a3-9687bd8c629a None None] MFA Rules not processed for user `835cbc13756b41b598142df3e35f5492`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:35.006706 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-0cf9c2df-0cb7-4ed9-87a3-9687bd8c629a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:35.029166 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 871/1758] 10.222.0.58 () {58 vars in 1018 bytes} [Tue Jul 26 13:46:34 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:46:35.030174 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf9c2df-0cb7-4ed9-87a3-9687bd8c629a None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:35.034927 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 888/1759] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:46:34 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:46:35.035986 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caee455-44ee-4e75-9f54-77b13cd1b635 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:35.056485 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf9c2df-0cb7-4ed9-87a3-9687bd8c629a None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:35.062721 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61078d7-9cec-478e-a05a-fd4a84415eb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:35.063685 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61078d7-9cec-478e-a05a-fd4a84415eb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:35.064694 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61078d7-9cec-478e-a05a-fd4a84415eb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:35.065161 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61078d7-9cec-478e-a05a-fd4a84415eb0 admin admin] PATH_INFO: `/v3/projects/5b0bc0916d8845bbb8fa2983ecadf587/users/e915a05773e34697a7c0092e09520736/roles/6e0201a9a6484f7d8b112b1eafff70bd` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:35.067567 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caee455-44ee-4e75-9f54-77b13cd1b635 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:35.081161 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cffc85-b35d-4df4-818e-fc1c32e4b656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:35.082137 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cffc85-b35d-4df4-818e-fc1c32e4b656 admin admin] REQUEST_METHOD: `PUT` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:35.083870 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cffc85-b35d-4df4-818e-fc1c32e4b656 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:35.084844 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61078d7-9cec-478e-a05a-fd4a84415eb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b0bc0916d8845bbb8fa2983ecadf587, user_id=e915a05773e34697a7c0092e09520736, role_id=6e0201a9a6484f7d8b112b1eafff70bd)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:35.085394 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cffc85-b35d-4df4-818e-fc1c32e4b656 admin admin] PATH_INFO: `/v3/projects/25e4f8cfa1254b7da37d813deef79e58/users/a0832d7bf4bb4c1b8b3e6bd9c6d0b03e/roles/759f389b221049c4b01faf3b984cd8d7` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:35.088261 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61078d7-9cec-478e-a05a-fd4a84415eb0 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:35.109878 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cffc85-b35d-4df4-818e-fc1c32e4b656 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25e4f8cfa1254b7da37d813deef79e58, user_id=a0832d7bf4bb4c1b8b3e6bd9c6d0b03e, role_id=759f389b221049c4b01faf3b984cd8d7)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:35.120464 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cffc85-b35d-4df4-818e-fc1c32e4b656 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:35.135871 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:35.145674 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 889/1760] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 13:46:35 2022] PUT /identity/v3/projects/25e4f8cfa1254b7da37d813deef79e58/users/a0832d7bf4bb4c1b8b3e6bd9c6d0b03e/roles/759f389b221049c4b01faf3b984cd8d7 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:46:35.149588 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 872/1761] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 13:46:35 2022] PUT /identity/v3/projects/5b0bc0916d8845bbb8fa2983ecadf587/users/e915a05773e34697a7c0092e09520736/roles/6e0201a9a6484f7d8b112b1eafff70bd => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:46:35.153412 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61078d7-9cec-478e-a05a-fd4a84415eb0 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:35.156890 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cffc85-b35d-4df4-818e-fc1c32e4b656 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:35.159335 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-a0cffc85-b35d-4df4-818e-fc1c32e4b656 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:35.315939 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61078d7-9cec-478e-a05a-fd4a84415eb0 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:35.348342 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cffc85-b35d-4df4-818e-fc1c32e4b656 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:35.364873 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9011884d-1cf2-42d3-9c4e-938bdb79950b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '835cbc13756b41b598142df3e35f5492', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:35.366641 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9011884d-1cf2-42d3-9c4e-938bdb79950b service placement] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:35.369347 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9011884d-1cf2-42d3-9c4e-938bdb79950b service placement] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:35.369866 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9011884d-1cf2-42d3-9c4e-938bdb79950b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:35.396552 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9011884d-1cf2-42d3-9c4e-938bdb79950b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:35.408278 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9011884d-1cf2-42d3-9c4e-938bdb79950b service placement] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:35.445635 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 890/1762] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:46:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:46:35.448995 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a77054-df56-431c-b9cd-97bfe4da61d4 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:35.452905 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a77054-df56-431c-b9cd-97bfe4da61d4 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:35.453261 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a77054-df56-431c-b9cd-97bfe4da61d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:35.513336 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:35.540369 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07153c07-b792-4c31-bb11-15f094a00d1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:35.540561 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-68a77054-df56-431c-b9cd-97bfe4da61d4 None None] MFA Rules not processed for user `a0832d7bf4bb4c1b8b3e6bd9c6d0b03e`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:35.541018 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07153c07-b792-4c31-bb11-15f094a00d1a admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:35.541297 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07153c07-b792-4c31-bb11-15f094a00d1a admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:35.541767 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-68a77054-df56-431c-b9cd-97bfe4da61d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:35.546102 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07153c07-b792-4c31-bb11-15f094a00d1a admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:35.546909 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07153c07-b792-4c31-bb11-15f094a00d1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:35.548571 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07153c07-b792-4c31-bb11-15f094a00d1a admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:35.604660 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 873/1763] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:46:35 2022] GET /identity/v3/roles => generated 1420 bytes in 452 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 13:46:35.606275 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739e7504-f0c0-4033-abbe-cae617660115 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:35.606575 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739e7504-f0c0-4033-abbe-cae617660115 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:35.607931 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739e7504-f0c0-4033-abbe-cae617660115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:35.662134 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-739e7504-f0c0-4033-abbe-cae617660115 None None] MFA Rules not processed for user `0accc3551cd348e2a946bde38e4db40f`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:35.663869 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-739e7504-f0c0-4033-abbe-cae617660115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:35.746788 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 891/1764] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:46:35 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:46:35.747801 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a77054-df56-431c-b9cd-97bfe4da61d4 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:35.794570 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a77054-df56-431c-b9cd-97bfe4da61d4 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:35.810505 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becc3755-002e-49f0-a38d-4552dd72c74f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:35.814390 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becc3755-002e-49f0-a38d-4552dd72c74f admin admin] REQUEST_METHOD: `PUT` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:35.814707 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becc3755-002e-49f0-a38d-4552dd72c74f admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:35.814969 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becc3755-002e-49f0-a38d-4552dd72c74f admin admin] PATH_INFO: `/v3/projects/5b0bc0916d8845bbb8fa2983ecadf587/users/e915a05773e34697a7c0092e09520736/roles/6e0201a9a6484f7d8b112b1eafff70bd` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:35.822344 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becc3755-002e-49f0-a38d-4552dd72c74f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b0bc0916d8845bbb8fa2983ecadf587, user_id=e915a05773e34697a7c0092e09520736, role_id=6e0201a9a6484f7d8b112b1eafff70bd)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:35.828807 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 874/1765] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:46:35 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:46:35.829982 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739e7504-f0c0-4033-abbe-cae617660115 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:35.832524 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becc3755-002e-49f0-a38d-4552dd72c74f admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:35.870124 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 892/1766] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 13:46:35 2022] PUT /identity/v3/projects/5b0bc0916d8845bbb8fa2983ecadf587/users/e915a05773e34697a7c0092e09520736/roles/6e0201a9a6484f7d8b112b1eafff70bd => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:46:35.879163 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ece13c-d32d-49b4-8a77-88bc1321e9b7 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:35.879163 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ece13c-d32d-49b4-8a77-88bc1321e9b7 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:35.879163 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ece13c-d32d-49b4-8a77-88bc1321e9b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:35.879458 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739e7504-f0c0-4033-abbe-cae617660115 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:35.915850 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-d4ece13c-d32d-49b4-8a77-88bc1321e9b7 None None] MFA Rules not processed for user `0accc3551cd348e2a946bde38e4db40f`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:35.918811 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-d4ece13c-d32d-49b4-8a77-88bc1321e9b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:36.030048 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0307f4de-88fc-40d0-b3f2-2266a3df4ab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:36.031818 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0307f4de-88fc-40d0-b3f2-2266a3df4ab4 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:36.032280 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0307f4de-88fc-40d0-b3f2-2266a3df4ab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:36.032703 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0307f4de-88fc-40d0-b3f2-2266a3df4ab4 admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:36.033721 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0307f4de-88fc-40d0-b3f2-2266a3df4ab4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:36.043485 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0307f4de-88fc-40d0-b3f2-2266a3df4ab4 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:36.056832 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 893/1767] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:46:35 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:46:36.062208 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec6b141-61f8-4d35-bfb8-33ffc5a6f76e None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:36.062494 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec6b141-61f8-4d35-bfb8-33ffc5a6f76e None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:36.062783 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec6b141-61f8-4d35-bfb8-33ffc5a6f76e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:36.099861 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:36.100938 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 875/1768] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 13:46:35 2022] POST /identity/v3/projects => generated 365 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:46:36.102167 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0307f4de-88fc-40d0-b3f2-2266a3df4ab4 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:36.104907 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-0307f4de-88fc-40d0-b3f2-2266a3df4ab4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:36.112607 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-3ec6b141-61f8-4d35-bfb8-33ffc5a6f76e None None] MFA Rules not processed for user `e915a05773e34697a7c0092e09520736`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:36.113871 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-3ec6b141-61f8-4d35-bfb8-33ffc5a6f76e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:36.207636 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0307f4de-88fc-40d0-b3f2-2266a3df4ab4 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:36.213247 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2699bb57-2c53-4a63-b69c-59474a38b30b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:36.213998 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2699bb57-2c53-4a63-b69c-59474a38b30b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:36.214282 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2699bb57-2c53-4a63-b69c-59474a38b30b admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:36.214564 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2699bb57-2c53-4a63-b69c-59474a38b30b admin admin] PATH_INFO: `/v3/projects/c447c7416c734b038a91f2921d5da9f6` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:36.216002 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2699bb57-2c53-4a63-b69c-59474a38b30b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c447c7416c734b038a91f2921d5da9f6)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:36.218349 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2699bb57-2c53-4a63-b69c-59474a38b30b admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:36.230922 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:36.252943 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 894/1769] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:46:36 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:46:36.253664 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec6b141-61f8-4d35-bfb8-33ffc5a6f76e None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:36.255569 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-3ec6b141-61f8-4d35-bfb8-33ffc5a6f76e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:36.331771 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-2699bb57-2c53-4a63-b69c-59474a38b30b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c447c7416c734b038a91f2921d5da9f6', 'request_id': 'req-2699bb57-2c53-4a63-b69c-59474a38b30b'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:36.350244 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-2699bb57-2c53-4a63-b69c-59474a38b30b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c447c7416c734b038a91f2921d5da9f6', 'request_id': 'req-2699bb57-2c53-4a63-b69c-59474a38b30b'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:36.354073 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec6b141-61f8-4d35-bfb8-33ffc5a6f76e None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:36.365431 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-2699bb57-2c53-4a63-b69c-59474a38b30b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c447c7416c734b038a91f2921d5da9f6', 'request_id': 'req-2699bb57-2c53-4a63-b69c-59474a38b30b'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:36.402884 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-2699bb57-2c53-4a63-b69c-59474a38b30b admin admin] The token cache is being invalidate because project c447c7416c734b038a91f2921d5da9f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 13:46:36.403587 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-2699bb57-2c53-4a63-b69c-59474a38b30b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:36.405345 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-2699bb57-2c53-4a63-b69c-59474a38b30b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:36.414505 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 876/1770] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 13:46:36 2022] DELETE /identity/v3/projects/c447c7416c734b038a91f2921d5da9f6 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:46:36.416062 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2699bb57-2c53-4a63-b69c-59474a38b30b admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:36.423238 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-2699bb57-2c53-4a63-b69c-59474a38b30b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:36.457898 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91f8fa7-88d7-4483-b632-35ab51309b76 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c69161cd520b4b29bf50f3bf2f1c033b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:36.458472 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91f8fa7-88d7-4483-b632-35ab51309b76 service nova] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:36.458755 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91f8fa7-88d7-4483-b632-35ab51309b76 service nova] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:36.459033 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91f8fa7-88d7-4483-b632-35ab51309b76 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:36.461367 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-a91f8fa7-88d7-4483-b632-35ab51309b76 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:36.535376 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:36.593063 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91f8fa7-88d7-4483-b632-35ab51309b76 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:36.595341 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91f8fa7-88d7-4483-b632-35ab51309b76 service nova] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:36.621999 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 895/1771] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:46:36 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:46:36.626251 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2699bb57-2c53-4a63-b69c-59474a38b30b admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:36.630928 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91f8fa7-88d7-4483-b632-35ab51309b76 service nova] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:36.641081 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b228723-4288-4db0-9b5c-2500c311cc91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:36.641842 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b228723-4288-4db0-9b5c-2500c311cc91 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:36.642121 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b228723-4288-4db0-9b5c-2500c311cc91 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:36.642399 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b228723-4288-4db0-9b5c-2500c311cc91 admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:36.643008 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-a91f8fa7-88d7-4483-b632-35ab51309b76 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:36.643120 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b228723-4288-4db0-9b5c-2500c311cc91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:36.645642 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b228723-4288-4db0-9b5c-2500c311cc91 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:36.700873 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 877/1772] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 13:46:36 2022] POST /identity/v3/users => generated 289 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:46:36.704124 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b228723-4288-4db0-9b5c-2500c311cc91 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:36.708239 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-3b228723-4288-4db0-9b5c-2500c311cc91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:36.739403 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b228723-4288-4db0-9b5c-2500c311cc91 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:36.754137 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc709ffd-af94-406a-9804-e57aa7b7a0c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:36.754870 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc709ffd-af94-406a-9804-e57aa7b7a0c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:36.755154 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc709ffd-af94-406a-9804-e57aa7b7a0c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:36.755436 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc709ffd-af94-406a-9804-e57aa7b7a0c5 admin admin] PATH_INFO: `/v3/users/fb001c7fda1b4bd492c981b7298c7239` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:36.766963 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc709ffd-af94-406a-9804-e57aa7b7a0c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb001c7fda1b4bd492c981b7298c7239)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:36.773265 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc709ffd-af94-406a-9804-e57aa7b7a0c5 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:36.814322 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91f8fa7-88d7-4483-b632-35ab51309b76 service nova] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:36.819015 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec59d28-9b01-48b3-8e89-e8190b511f83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:36.819763 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec59d28-9b01-48b3-8e89-e8190b511f83 service cinder] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:36.820045 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec59d28-9b01-48b3-8e89-e8190b511f83 service cinder] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:36.820319 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec59d28-9b01-48b3-8e89-e8190b511f83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:36.823145 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-6ec59d28-9b01-48b3-8e89-e8190b511f83 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:36.978878 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-dc709ffd-af94-406a-9804-e57aa7b7a0c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb001c7fda1b4bd492c981b7298c7239'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:36.979792 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-dc709ffd-af94-406a-9804-e57aa7b7a0c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb001c7fda1b4bd492c981b7298c7239'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:36.990962 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec59d28-9b01-48b3-8e89-e8190b511f83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:36.992595 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec59d28-9b01-48b3-8e89-e8190b511f83 service cinder] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:37.012271 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-dc709ffd-af94-406a-9804-e57aa7b7a0c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb001c7fda1b4bd492c981b7298c7239'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:37.013279 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-dc709ffd-af94-406a-9804-e57aa7b7a0c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb001c7fda1b4bd492c981b7298c7239'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:37.026284 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 896/1773] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:46:36 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:46:37.045039 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec59d28-9b01-48b3-8e89-e8190b511f83 service cinder] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:37.054122 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-dc709ffd-af94-406a-9804-e57aa7b7a0c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb001c7fda1b4bd492c981b7298c7239'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:37.066655 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=78,keystone:DELETE=17,keystone:INSERT=5 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:37.077808 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec59d28-9b01-48b3-8e89-e8190b511f83 service cinder] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:37.087532 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 878/1774] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 13:46:36 2022] DELETE /identity/v3/users/fb001c7fda1b4bd492c981b7298c7239 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:46:37.090577 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98129b40-789b-40a6-b5d8-2d227c813954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:37.091484 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98129b40-789b-40a6-b5d8-2d227c813954 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:37.091774 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98129b40-789b-40a6-b5d8-2d227c813954 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:37.092050 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98129b40-789b-40a6-b5d8-2d227c813954 admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:37.092850 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98129b40-789b-40a6-b5d8-2d227c813954 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:37.097103 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98129b40-789b-40a6-b5d8-2d227c813954 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:37.141178 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 897/1775] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:46:37 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:46:37.143122 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc709ffd-af94-406a-9804-e57aa7b7a0c5 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:37.144809 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-dc709ffd-af94-406a-9804-e57aa7b7a0c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:37.200537 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98129b40-789b-40a6-b5d8-2d227c813954 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:37.202396 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-98129b40-789b-40a6-b5d8-2d227c813954 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:37.228281 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98129b40-789b-40a6-b5d8-2d227c813954 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:37.411168 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc709ffd-af94-406a-9804-e57aa7b7a0c5 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:37.426243 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4498d8-10b1-4751-a041-f10d0c0e5718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:37.426963 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4498d8-10b1-4751-a041-f10d0c0e5718 admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:37.427249 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4498d8-10b1-4751-a041-f10d0c0e5718 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:37.427535 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4498d8-10b1-4751-a041-f10d0c0e5718 admin admin] PATH_INFO: `/v3/projects/df0f039f83384f83ae5d6f9e96d168ff/users/ad97ae3435234e57abc371244a8758b1/roles/6e0201a9a6484f7d8b112b1eafff70bd` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:37.439953 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ace727-7a0c-47ef-9780-92a47484f7b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:37.440493 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ace727-7a0c-47ef-9780-92a47484f7b8 service cinder] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:37.440784 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ace727-7a0c-47ef-9780-92a47484f7b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:37.441067 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ace727-7a0c-47ef-9780-92a47484f7b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:37.450056 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-d4ace727-7a0c-47ef-9780-92a47484f7b8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:37.481175 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4498d8-10b1-4751-a041-f10d0c0e5718 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df0f039f83384f83ae5d6f9e96d168ff, user_id=ad97ae3435234e57abc371244a8758b1, role_id=6e0201a9a6484f7d8b112b1eafff70bd)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:37.483865 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4498d8-10b1-4751-a041-f10d0c0e5718 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:37.515498 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:37.528508 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 879/1776] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 13:46:37 2022] PUT /identity/v3/projects/df0f039f83384f83ae5d6f9e96d168ff/users/ad97ae3435234e57abc371244a8758b1/roles/6e0201a9a6484f7d8b112b1eafff70bd => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:46:37.533253 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4498d8-10b1-4751-a041-f10d0c0e5718 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:37.706879 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ace727-7a0c-47ef-9780-92a47484f7b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:37.708647 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ace727-7a0c-47ef-9780-92a47484f7b8 service cinder] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:37.718677 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4498d8-10b1-4751-a041-f10d0c0e5718 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:37.731716 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cacd95c-626b-465a-9e55-098ef6f1a87d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:37.733651 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 898/1777] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:46:37 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 533 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 13:46:37.734507 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cacd95c-626b-465a-9e55-098ef6f1a87d admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:37.734928 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ace727-7a0c-47ef-9780-92a47484f7b8 service cinder] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:37.735974 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cacd95c-626b-465a-9e55-098ef6f1a87d admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:37.736262 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cacd95c-626b-465a-9e55-098ef6f1a87d admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:37.737093 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cacd95c-626b-465a-9e55-098ef6f1a87d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:37.737293 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-d4ace727-7a0c-47ef-9780-92a47484f7b8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:37.741270 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cacd95c-626b-465a-9e55-098ef6f1a87d admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:37.768836 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 880/1778] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:46:37 2022] GET /identity/v3/roles => generated 1420 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 13:46:37.770221 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d048e725-9e6f-4686-aab0-8dd4506ad7a5 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:37.770509 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d048e725-9e6f-4686-aab0-8dd4506ad7a5 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:37.770789 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d048e725-9e6f-4686-aab0-8dd4506ad7a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:37.787163 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ace727-7a0c-47ef-9780-92a47484f7b8 service cinder] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:37.798751 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53658762-0f96-4a08-a86d-61108c860686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:37.800286 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53658762-0f96-4a08-a86d-61108c860686 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:37.800654 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53658762-0f96-4a08-a86d-61108c860686 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:37.801574 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53658762-0f96-4a08-a86d-61108c860686 admin admin] PATH_INFO: `/v3/users/66217dc1b3f84d82a5300ee1ff983da4` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:37.808994 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53658762-0f96-4a08-a86d-61108c860686 admin admin] RBAC: Authorizing `identity:delete_user(user_id=66217dc1b3f84d82a5300ee1ff983da4)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:37.811630 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53658762-0f96-4a08-a86d-61108c860686 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:37.832436 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-d048e725-9e6f-4686-aab0-8dd4506ad7a5 None None] MFA Rules not processed for user `1747d40a285942eb9b4d5bc14136d88a`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:37.833126 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-d048e725-9e6f-4686-aab0-8dd4506ad7a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:37.946129 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-53658762-0f96-4a08-a86d-61108c860686 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66217dc1b3f84d82a5300ee1ff983da4'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:37.972602 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-53658762-0f96-4a08-a86d-61108c860686 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66217dc1b3f84d82a5300ee1ff983da4'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:37.998559 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-53658762-0f96-4a08-a86d-61108c860686 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66217dc1b3f84d82a5300ee1ff983da4'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:38.004179 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-53658762-0f96-4a08-a86d-61108c860686 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66217dc1b3f84d82a5300ee1ff983da4'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:38.006275 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-53658762-0f96-4a08-a86d-61108c860686 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66217dc1b3f84d82a5300ee1ff983da4'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:38.058719 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 881/1779] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:46:37 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:46:38.059619 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d048e725-9e6f-4686-aab0-8dd4506ad7a5 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:38.061258 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 899/1780] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 13:46:37 2022] DELETE /identity/v3/users/66217dc1b3f84d82a5300ee1ff983da4 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:46:38.071799 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-d048e725-9e6f-4686-aab0-8dd4506ad7a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:38.120425 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d048e725-9e6f-4686-aab0-8dd4506ad7a5 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:38.217400 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:38.360349 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6505e0c-bad9-4189-84a4-df9e3962c6b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:38.362066 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6505e0c-bad9-4189-84a4-df9e3962c6b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:38.362358 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6505e0c-bad9-4189-84a4-df9e3962c6b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:38.362642 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6505e0c-bad9-4189-84a4-df9e3962c6b8 admin admin] PATH_INFO: `/v3/projects/df0f039f83384f83ae5d6f9e96d168ff/users/ad97ae3435234e57abc371244a8758b1/roles/6e0201a9a6484f7d8b112b1eafff70bd` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:38.369757 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6505e0c-bad9-4189-84a4-df9e3962c6b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df0f039f83384f83ae5d6f9e96d168ff, user_id=ad97ae3435234e57abc371244a8758b1, role_id=6e0201a9a6484f7d8b112b1eafff70bd)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:38.379117 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6505e0c-bad9-4189-84a4-df9e3962c6b8 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:38.394605 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d050d6-4965-4bbf-b900-04e0f5995b38 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:38.394974 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d050d6-4965-4bbf-b900-04e0f5995b38 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:38.395365 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d050d6-4965-4bbf-b900-04e0f5995b38 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:38.397695 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 900/1781] 10.222.0.58 () {54 vars in 912 bytes} [Tue Jul 26 13:46:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 13:46:38.407473 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5abb01-7f41-44cc-b34a-5b77be4a4258 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:38.408072 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5abb01-7f41-44cc-b34a-5b77be4a4258 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:38.408423 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5abb01-7f41-44cc-b34a-5b77be4a4258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:38.425364 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 882/1782] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 13:46:38 2022] PUT /identity/v3/projects/df0f039f83384f83ae5d6f9e96d168ff/users/ad97ae3435234e57abc371244a8758b1/roles/6e0201a9a6484f7d8b112b1eafff70bd => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:46:38.436489 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aec1c5-99c6-4e06-9093-cb9ca0e36219 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:38.436812 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aec1c5-99c6-4e06-9093-cb9ca0e36219 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:38.437115 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aec1c5-99c6-4e06-9093-cb9ca0e36219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:38.477828 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-fe5abb01-7f41-44cc-b34a-5b77be4a4258 None None] MFA Rules not processed for user `3223a78866d94c2ba66402911c2dfae9`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:38.482276 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-fe5abb01-7f41-44cc-b34a-5b77be4a4258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:38.495754 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-32aec1c5-99c6-4e06-9093-cb9ca0e36219 None None] MFA Rules not processed for user `ad97ae3435234e57abc371244a8758b1`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:38.497002 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-32aec1c5-99c6-4e06-9093-cb9ca0e36219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:38.677634 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 901/1783] 10.222.0.58 () {58 vars in 1009 bytes} [Tue Jul 26 13:46:38 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 13:46:38.688234 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d012d6cf-4b0b-4af3-beba-6db8a38df1f7 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:38.688525 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d012d6cf-4b0b-4af3-beba-6db8a38df1f7 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:38.688805 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d012d6cf-4b0b-4af3-beba-6db8a38df1f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:38.764096 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 883/1784] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:46:38 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:46:38.780012 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aec1c5-99c6-4e06-9093-cb9ca0e36219 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:38.781833 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-32aec1c5-99c6-4e06-9093-cb9ca0e36219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:38.849269 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-d012d6cf-4b0b-4af3-beba-6db8a38df1f7 None None] MFA Rules not processed for user `b64bd7b183d24d8c9037b261c7884810`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:38.850594 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-d012d6cf-4b0b-4af3-beba-6db8a38df1f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:38.909276 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:38.981749 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aec1c5-99c6-4e06-9093-cb9ca0e36219 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:38.990550 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d4238f-3fd5-4b01-a001-88bc700330db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:38.990909 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d4238f-3fd5-4b01-a001-88bc700330db service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:38.991196 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d4238f-3fd5-4b01-a001-88bc700330db service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:38.991471 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d4238f-3fd5-4b01-a001-88bc700330db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:39.011189 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d4238f-3fd5-4b01-a001-88bc700330db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:39.012475 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d4238f-3fd5-4b01-a001-88bc700330db service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:39.024072 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 884/1785] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:46:38 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 13:46:39.024944 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d4238f-3fd5-4b01-a001-88bc700330db service cinder] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:39.026498 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-b0d4238f-3fd5-4b01-a001-88bc700330db service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:39.027584 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 902/1786] 10.222.0.58 () {58 vars in 1009 bytes} [Tue Jul 26 13:46:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:46:39.113092 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d012d6cf-4b0b-4af3-beba-6db8a38df1f7 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:39.114701 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-d012d6cf-4b0b-4af3-beba-6db8a38df1f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:39.133150 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:39.137078 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d012d6cf-4b0b-4af3-beba-6db8a38df1f7 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:39.150192 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c88319-ece1-4cb0-b990-d3dc1301777c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:39.151107 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c88319-ece1-4cb0-b990-d3dc1301777c service neutron] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:39.152666 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c88319-ece1-4cb0-b990-d3dc1301777c service neutron] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:39.153137 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c88319-ece1-4cb0-b990-d3dc1301777c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:39.171810 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c88319-ece1-4cb0-b990-d3dc1301777c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:39.177306 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c88319-ece1-4cb0-b990-d3dc1301777c service neutron] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:39.186107 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d4238f-3fd5-4b01-a001-88bc700330db service cinder] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:39.216051 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea5f352-0fb1-40d1-8250-ff57b74b0b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:39.218716 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 903/1787] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:46:39 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:46:39.219626 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea5f352-0fb1-40d1-8250-ff57b74b0b0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:39.220113 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea5f352-0fb1-40d1-8250-ff57b74b0b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:39.220794 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea5f352-0fb1-40d1-8250-ff57b74b0b0c admin admin] PATH_INFO: `/v3/projects/8b3c5544eaf548beb951ab4548e6dbd0` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:39.225077 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea5f352-0fb1-40d1-8250-ff57b74b0b0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b3c5544eaf548beb951ab4548e6dbd0)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:39.227474 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea5f352-0fb1-40d1-8250-ff57b74b0b0c admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:39.287950 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c88319-ece1-4cb0-b990-d3dc1301777c service neutron] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:39.289658 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-d5c88319-ece1-4cb0-b990-d3dc1301777c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:39.418448 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-5ea5f352-0fb1-40d1-8250-ff57b74b0b0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b3c5544eaf548beb951ab4548e6dbd0', 'request_id': 'req-5ea5f352-0fb1-40d1-8250-ff57b74b0b0c'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:39.450251 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-5ea5f352-0fb1-40d1-8250-ff57b74b0b0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b3c5544eaf548beb951ab4548e6dbd0', 'request_id': 'req-5ea5f352-0fb1-40d1-8250-ff57b74b0b0c'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:39.474911 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-5ea5f352-0fb1-40d1-8250-ff57b74b0b0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b3c5544eaf548beb951ab4548e6dbd0', 'request_id': 'req-5ea5f352-0fb1-40d1-8250-ff57b74b0b0c'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:39.523136 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-5ea5f352-0fb1-40d1-8250-ff57b74b0b0c admin admin] The token cache is being invalidate because project 8b3c5544eaf548beb951ab4548e6dbd0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 13:46:39.523346 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-5ea5f352-0fb1-40d1-8250-ff57b74b0b0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:39.525802 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-5ea5f352-0fb1-40d1-8250-ff57b74b0b0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:39.530968 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 885/1788] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 13:46:39 2022] DELETE /identity/v3/projects/8b3c5544eaf548beb951ab4548e6dbd0 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:46:39.537231 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c7eb04-5c63-4ae1-8997-d4174f95a645 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:39.537596 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c7eb04-5c63-4ae1-8997-d4174f95a645 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:39.537893 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c7eb04-5c63-4ae1-8997-d4174f95a645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:39.592583 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c88319-ece1-4cb0-b990-d3dc1301777c service neutron] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:39.605753 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-b6c7eb04-5c63-4ae1-8997-d4174f95a645 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:39.607243 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-b6c7eb04-5c63-4ae1-8997-d4174f95a645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:39.787775 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 886/1789] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:46:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 13:46:39.797688 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cd0c32-b98f-4483-bbfd-bfcc8bbafac4 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:39.798220 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cd0c32-b98f-4483-bbfd-bfcc8bbafac4 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:39.798615 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cd0c32-b98f-4483-bbfd-bfcc8bbafac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:39.809434 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5c93df-f9ea-4110-a4c2-df68d17afac3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab2819bac844163ae4305e4f2186605', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:39.810292 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5c93df-f9ea-4110-a4c2-df68d17afac3 service swift] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:39.810828 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5c93df-f9ea-4110-a4c2-df68d17afac3 service swift] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:39.811206 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5c93df-f9ea-4110-a4c2-df68d17afac3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:39.817199 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-5f5c93df-f9ea-4110-a4c2-df68d17afac3 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:39.859285 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-b4cd0c32-b98f-4483-bbfd-bfcc8bbafac4 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:39.863763 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-b4cd0c32-b98f-4483-bbfd-bfcc8bbafac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:39.894060 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:39.894952 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 887/1790] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:46:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:46:39.901442 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cd0c32-b98f-4483-bbfd-bfcc8bbafac4 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:39.934619 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cd0c32-b98f-4483-bbfd-bfcc8bbafac4 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:39.962602 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0bb3f9-eb2d-41e0-bf46-acf10a69e7ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:39.963228 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0bb3f9-eb2d-41e0-bf46-acf10a69e7ee admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:39.963508 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0bb3f9-eb2d-41e0-bf46-acf10a69e7ee admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:39.963782 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0bb3f9-eb2d-41e0-bf46-acf10a69e7ee admin admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:39.964489 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0bb3f9-eb2d-41e0-bf46-acf10a69e7ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:46:39.965330 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0bb3f9-eb2d-41e0-bf46-acf10a69e7ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:39.966423 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0bb3f9-eb2d-41e0-bf46-acf10a69e7ee admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:40.010294 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5c93df-f9ea-4110-a4c2-df68d17afac3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:40.018069 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5c93df-f9ea-4110-a4c2-df68d17afac3 service swift] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:40.048371 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 888/1791] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Jul 26 13:46:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:46:40.049066 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0bb3f9-eb2d-41e0-bf46-acf10a69e7ee admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:40.066518 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 904/1792] 10.222.0.58 () {60 vars in 1384 bytes} [Tue Jul 26 13:46:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 779 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 13:46:40.089274 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0bb3f9-eb2d-41e0-bf46-acf10a69e7ee admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:40.116995 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88be887-f77d-4b9c-8534-d477eefd74a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:40.117883 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88be887-f77d-4b9c-8534-d477eefd74a6 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:40.118273 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88be887-f77d-4b9c-8534-d477eefd74a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:40.118650 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88be887-f77d-4b9c-8534-d477eefd74a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:40.120158 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88be887-f77d-4b9c-8534-d477eefd74a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:40.122856 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88be887-f77d-4b9c-8534-d477eefd74a6 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:40.174836 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5c93df-f9ea-4110-a4c2-df68d17afac3 service swift] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:40.177021 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-5f5c93df-f9ea-4110-a4c2-df68d17afac3 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:40.210751 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5c93df-f9ea-4110-a4c2-df68d17afac3 service swift] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:40.233279 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c2d2cb-b69e-45b0-8977-bc291a29cdca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:40.234544 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c2d2cb-b69e-45b0-8977-bc291a29cdca admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:40.234975 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c2d2cb-b69e-45b0-8977-bc291a29cdca admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:40.235429 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c2d2cb-b69e-45b0-8977-bc291a29cdca admin admin] PATH_INFO: `/v3/users/a0832d7bf4bb4c1b8b3e6bd9c6d0b03e` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:40.237396 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c2d2cb-b69e-45b0-8977-bc291a29cdca admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0832d7bf4bb4c1b8b3e6bd9c6d0b03e)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:40.245104 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c2d2cb-b69e-45b0-8977-bc291a29cdca admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:40.294234 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 889/1793] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 13:46:40 2022] POST /identity/v3/projects => generated 389 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:46:40.295710 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88be887-f77d-4b9c-8534-d477eefd74a6 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:40.479308 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:40.480135 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-95c2d2cb-b69e-45b0-8977-bc291a29cdca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0832d7bf4bb4c1b8b3e6bd9c6d0b03e'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:40.504655 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-95c2d2cb-b69e-45b0-8977-bc291a29cdca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0832d7bf4bb4c1b8b3e6bd9c6d0b03e'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:40.538857 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-95c2d2cb-b69e-45b0-8977-bc291a29cdca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0832d7bf4bb4c1b8b3e6bd9c6d0b03e'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:40.538857 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-95c2d2cb-b69e-45b0-8977-bc291a29cdca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0832d7bf4bb4c1b8b3e6bd9c6d0b03e'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:40.538857 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-95c2d2cb-b69e-45b0-8977-bc291a29cdca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0832d7bf4bb4c1b8b3e6bd9c6d0b03e'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:40.636021 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88be887-f77d-4b9c-8534-d477eefd74a6 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:40.651667 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 905/1794] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 13:46:40 2022] DELETE /identity/v3/users/a0832d7bf4bb4c1b8b3e6bd9c6d0b03e => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 13:46:40.656647 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fa15d8-7c30-42ea-b15e-94659599c6f4 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:40.657600 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fa15d8-7c30-42ea-b15e-94659599c6f4 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:40.657886 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fa15d8-7c30-42ea-b15e-94659599c6f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:40.661746 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aeee7b-597b-48e6-9265-d1027f6809a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:40.662461 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aeee7b-597b-48e6-9265-d1027f6809a6 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:40.662749 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aeee7b-597b-48e6-9265-d1027f6809a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:40.663025 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aeee7b-597b-48e6-9265-d1027f6809a6 admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:40.663715 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65aeee7b-597b-48e6-9265-d1027f6809a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:40.665691 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65aeee7b-597b-48e6-9265-d1027f6809a6 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:40.760610 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-a8fa15d8-7c30-42ea-b15e-94659599c6f4 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:40.762162 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-a8fa15d8-7c30-42ea-b15e-94659599c6f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:40.791793 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 890/1795] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 13:46:40 2022] POST /identity/v3/users => generated 295 bytes in 497 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:46:40.809507 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aeee7b-597b-48e6-9265-d1027f6809a6 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:40.817025 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-65aeee7b-597b-48e6-9265-d1027f6809a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:40.857654 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aeee7b-597b-48e6-9265-d1027f6809a6 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:41.079649 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 906/1796] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:46:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 424 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:46:41.080535 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fa15d8-7c30-42ea-b15e-94659599c6f4 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:41.094871 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-a8fa15d8-7c30-42ea-b15e-94659599c6f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:41.136865 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fa15d8-7c30-42ea-b15e-94659599c6f4 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:41.145952 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639371d7-4f7d-48ad-82bf-d047fd473a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:41.147054 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639371d7-4f7d-48ad-82bf-d047fd473a04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:41.147347 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639371d7-4f7d-48ad-82bf-d047fd473a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:41.147716 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639371d7-4f7d-48ad-82bf-d047fd473a04 admin admin] PATH_INFO: `/v3/projects/c057f946baaf43c1a6d147d846418604` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:41.151251 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7764c2a7-7051-438d-bb70-9f0300616088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:41.152403 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639371d7-4f7d-48ad-82bf-d047fd473a04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c057f946baaf43c1a6d147d846418604)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:41.153037 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7764c2a7-7051-438d-bb70-9f0300616088 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:41.153354 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7764c2a7-7051-438d-bb70-9f0300616088 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:41.155141 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7764c2a7-7051-438d-bb70-9f0300616088 admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:41.156581 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7764c2a7-7051-438d-bb70-9f0300616088 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:41.158849 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7764c2a7-7051-438d-bb70-9f0300616088 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:41.160861 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639371d7-4f7d-48ad-82bf-d047fd473a04 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:41.195581 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 907/1797] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:46:41 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:46:41.197127 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f715f88-77e9-477d-b602-085ed96c46c2 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:41.197421 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f715f88-77e9-477d-b602-085ed96c46c2 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:41.197729 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f715f88-77e9-477d-b602-085ed96c46c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:41.242835 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-8f715f88-77e9-477d-b602-085ed96c46c2 None None] MFA Rules not processed for user `56db6603f359477885b5151dec71c096`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:41.244950 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-8f715f88-77e9-477d-b602-085ed96c46c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:41.301763 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-639371d7-4f7d-48ad-82bf-d047fd473a04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c057f946baaf43c1a6d147d846418604', 'request_id': 'req-639371d7-4f7d-48ad-82bf-d047fd473a04'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:41.328684 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-639371d7-4f7d-48ad-82bf-d047fd473a04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c057f946baaf43c1a6d147d846418604', 'request_id': 'req-639371d7-4f7d-48ad-82bf-d047fd473a04'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:41.352065 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:41.368418 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-639371d7-4f7d-48ad-82bf-d047fd473a04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c057f946baaf43c1a6d147d846418604', 'request_id': 'req-639371d7-4f7d-48ad-82bf-d047fd473a04'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:41.442594 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-639371d7-4f7d-48ad-82bf-d047fd473a04 admin admin] The token cache is being invalidate because project c057f946baaf43c1a6d147d846418604 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 13:46:41.442883 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-639371d7-4f7d-48ad-82bf-d047fd473a04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:41.449294 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-639371d7-4f7d-48ad-82bf-d047fd473a04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:41.458292 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 891/1798] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 13:46:40 2022] DELETE /identity/v3/projects/c057f946baaf43c1a6d147d846418604 => generated 0 bytes in 649 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:46:41.461379 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639371d7-4f7d-48ad-82bf-d047fd473a04 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:41.464007 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-639371d7-4f7d-48ad-82bf-d047fd473a04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:41.514222 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 908/1799] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:46:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:46:41.514931 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f715f88-77e9-477d-b602-085ed96c46c2 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:41.519544 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-8f715f88-77e9-477d-b602-085ed96c46c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:41.686724 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639371d7-4f7d-48ad-82bf-d047fd473a04 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:41.701857 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a0cc34-5aac-4686-8b17-ec0858dc0ed4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:41.702588 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a0cc34-5aac-4686-8b17-ec0858dc0ed4 service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:41.702876 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a0cc34-5aac-4686-8b17-ec0858dc0ed4 service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:41.703169 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a0cc34-5aac-4686-8b17-ec0858dc0ed4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:41.711066 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-49a0cc34-5aac-4686-8b17-ec0858dc0ed4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:41.724058 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:41.863791 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f715f88-77e9-477d-b602-085ed96c46c2 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:41.886410 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9cf992-1286-4c67-9dd9-34fe3c901a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:41.887086 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9cf992-1286-4c67-9dd9-34fe3c901a6b admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:41.887382 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9cf992-1286-4c67-9dd9-34fe3c901a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:41.887652 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9cf992-1286-4c67-9dd9-34fe3c901a6b admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:41.888429 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9cf992-1286-4c67-9dd9-34fe3c901a6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:41.895681 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9cf992-1286-4c67-9dd9-34fe3c901a6b admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:41.932341 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 909/1800] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:46:41 2022] GET /identity/v3/roles => generated 1420 bytes in 414 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:46:41.932341 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9cf992-1286-4c67-9dd9-34fe3c901a6b admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:41.936748 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-da9cf992-1286-4c67-9dd9-34fe3c901a6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:41.986644 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9cf992-1286-4c67-9dd9-34fe3c901a6b admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:41.993969 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a0cc34-5aac-4686-8b17-ec0858dc0ed4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:41.995816 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a0cc34-5aac-4686-8b17-ec0858dc0ed4 service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:42.006563 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9717db98-374b-4830-8462-df3ef8e6b4dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:42.009219 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9717db98-374b-4830-8462-df3ef8e6b4dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:42.012316 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9717db98-374b-4830-8462-df3ef8e6b4dd admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:42.012316 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9717db98-374b-4830-8462-df3ef8e6b4dd admin admin] PATH_INFO: `/v3/users/85f4eb91a7e843e284628d423cf22f7a` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:42.017791 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9717db98-374b-4830-8462-df3ef8e6b4dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=85f4eb91a7e843e284628d423cf22f7a)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:42.027606 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9717db98-374b-4830-8462-df3ef8e6b4dd admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:42.048148 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 892/1801] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:46:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 589 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:46:42.049645 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a0cc34-5aac-4686-8b17-ec0858dc0ed4 service neutron] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:42.062662 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a0cc34-5aac-4686-8b17-ec0858dc0ed4 service neutron] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:42.070650 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f87a37-0159-4f80-ac8a-7892a90c1f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:42.071688 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f87a37-0159-4f80-ac8a-7892a90c1f67 admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:42.072120 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f87a37-0159-4f80-ac8a-7892a90c1f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:42.072536 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f87a37-0159-4f80-ac8a-7892a90c1f67 admin admin] PATH_INFO: `/v3/projects/0979dcd11d394a1d8efa7c9d6dfd3e03/users/6128975041594653b5e56cb99ab5bace/roles/759f389b221049c4b01faf3b984cd8d7` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:42.098522 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f87a37-0159-4f80-ac8a-7892a90c1f67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0979dcd11d394a1d8efa7c9d6dfd3e03, user_id=6128975041594653b5e56cb99ab5bace, role_id=759f389b221049c4b01faf3b984cd8d7)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:42.101435 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f87a37-0159-4f80-ac8a-7892a90c1f67 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:42.147185 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 893/1802] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 13:46:42 2022] PUT /identity/v3/projects/0979dcd11d394a1d8efa7c9d6dfd3e03/users/6128975041594653b5e56cb99ab5bace/roles/759f389b221049c4b01faf3b984cd8d7 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:46:42.150055 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee36f34-e027-4b5d-9e05-d27fd8a4c755 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:42.150522 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee36f34-e027-4b5d-9e05-d27fd8a4c755 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:42.150864 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee36f34-e027-4b5d-9e05-d27fd8a4c755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:42.205089 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-9717db98-374b-4830-8462-df3ef8e6b4dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85f4eb91a7e843e284628d423cf22f7a'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:42.222195 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-9717db98-374b-4830-8462-df3ef8e6b4dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85f4eb91a7e843e284628d423cf22f7a'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:42.243806 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-dee36f34-e027-4b5d-9e05-d27fd8a4c755 None None] MFA Rules not processed for user `6128975041594653b5e56cb99ab5bace`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:42.245206 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-dee36f34-e027-4b5d-9e05-d27fd8a4c755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:42.249158 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-9717db98-374b-4830-8462-df3ef8e6b4dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85f4eb91a7e843e284628d423cf22f7a'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:42.252281 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-9717db98-374b-4830-8462-df3ef8e6b4dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85f4eb91a7e843e284628d423cf22f7a'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:42.254478 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-9717db98-374b-4830-8462-df3ef8e6b4dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85f4eb91a7e843e284628d423cf22f7a'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:42.337967 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 910/1803] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 13:46:41 2022] DELETE /identity/v3/users/85f4eb91a7e843e284628d423cf22f7a => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:46:42.544860 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 894/1804] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:46:42 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 396 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:46:42.545427 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9717db98-374b-4830-8462-df3ef8e6b4dd admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:42.547565 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-9717db98-374b-4830-8462-df3ef8e6b4dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:42.581552 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb95b550-c615-44eb-a144-56a49626e648 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:42.581892 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb95b550-c615-44eb-a144-56a49626e648 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:42.582181 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb95b550-c615-44eb-a144-56a49626e648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:42.679264 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-eb95b550-c615-44eb-a144-56a49626e648 None None] MFA Rules not processed for user `0accc3551cd348e2a946bde38e4db40f`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:42.686221 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-eb95b550-c615-44eb-a144-56a49626e648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:42.777236 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:42.868819 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 895/1805] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:46:42 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:46:42.877724 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9717db98-374b-4830-8462-df3ef8e6b4dd admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:42.916218 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d22374-6033-44a5-9234-a34fdfa77c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:42.916893 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d22374-6033-44a5-9234-a34fdfa77c2c admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:42.917160 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d22374-6033-44a5-9234-a34fdfa77c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:42.917425 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d22374-6033-44a5-9234-a34fdfa77c2c admin admin] PATH_INFO: `/v3/projects` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:42.918394 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d22374-6033-44a5-9234-a34fdfa77c2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:42.931810 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d22374-6033-44a5-9234-a34fdfa77c2c admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:43.061046 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 911/1806] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 13:46:42 2022] POST /identity/v3/projects => generated 389 bytes in 517 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:46:43.065056 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb95b550-c615-44eb-a144-56a49626e648 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:43.085140 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb95b550-c615-44eb-a144-56a49626e648 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:43.135729 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:43.267670 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d22374-6033-44a5-9234-a34fdfa77c2c admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:43.269988 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-14d22374-6033-44a5-9234-a34fdfa77c2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:43.357016 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffbde49-23c6-41fd-ac65-fc153816e783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:43.360088 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffbde49-23c6-41fd-ac65-fc153816e783 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:43.360534 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffbde49-23c6-41fd-ac65-fc153816e783 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:43.362041 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffbde49-23c6-41fd-ac65-fc153816e783 admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:43.363601 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffbde49-23c6-41fd-ac65-fc153816e783 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:43.367575 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffbde49-23c6-41fd-ac65-fc153816e783 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:43.426547 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 896/1807] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 13:46:43 2022] POST /identity/v3/users => generated 301 bytes in 362 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:46:43.432811 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffbde49-23c6-41fd-ac65-fc153816e783 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:43.458085 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffbde49-23c6-41fd-ac65-fc153816e783 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:43.471179 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eb83f7-54c2-47d8-9083-7b3a1babd856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:43.472305 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eb83f7-54c2-47d8-9083-7b3a1babd856 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:43.472895 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eb83f7-54c2-47d8-9083-7b3a1babd856 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:43.473314 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eb83f7-54c2-47d8-9083-7b3a1babd856 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:43.474210 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eb83f7-54c2-47d8-9083-7b3a1babd856 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:43.476273 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eb83f7-54c2-47d8-9083-7b3a1babd856 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:43.478549 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d22374-6033-44a5-9234-a34fdfa77c2c admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:43.501247 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1699d5a0-20bd-4b4f-8d0f-e7a4dae33289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:43.501868 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 897/1808] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:46:43 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:46:43.502033 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1699d5a0-20bd-4b4f-8d0f-e7a4dae33289 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:43.503626 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1699d5a0-20bd-4b4f-8d0f-e7a4dae33289 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:43.503912 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1699d5a0-20bd-4b4f-8d0f-e7a4dae33289 admin admin] PATH_INFO: `/v3/projects/25e4f8cfa1254b7da37d813deef79e58` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:43.505744 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eb83f7-54c2-47d8-9083-7b3a1babd856 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:43.511102 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1699d5a0-20bd-4b4f-8d0f-e7a4dae33289 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25e4f8cfa1254b7da37d813deef79e58)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:43.519175 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1699d5a0-20bd-4b4f-8d0f-e7a4dae33289 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:43.524443 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eb83f7-54c2-47d8-9083-7b3a1babd856 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:43.532199 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58efe06-4caf-4a86-adec-316706bde0c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:43.534230 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58efe06-4caf-4a86-adec-316706bde0c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:43.534764 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58efe06-4caf-4a86-adec-316706bde0c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:43.535227 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58efe06-4caf-4a86-adec-316706bde0c2 admin admin] PATH_INFO: `/v3/projects/64d5eff0ff75416dbd843d51ee3ecc3c/users/124b6809816b4653ae15e3f6711813e4/roles/6e0201a9a6484f7d8b112b1eafff70bd` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:43.566434 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58efe06-4caf-4a86-adec-316706bde0c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64d5eff0ff75416dbd843d51ee3ecc3c, user_id=124b6809816b4653ae15e3f6711813e4, role_id=6e0201a9a6484f7d8b112b1eafff70bd)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:43.575212 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58efe06-4caf-4a86-adec-316706bde0c2 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:43.608241 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 898/1809] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 13:46:43 2022] PUT /identity/v3/projects/64d5eff0ff75416dbd843d51ee3ecc3c/users/124b6809816b4653ae15e3f6711813e4/roles/6e0201a9a6484f7d8b112b1eafff70bd => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:46:43.612029 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58efe06-4caf-4a86-adec-316706bde0c2 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:43.629806 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-1699d5a0-20bd-4b4f-8d0f-e7a4dae33289 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25e4f8cfa1254b7da37d813deef79e58', 'request_id': 'req-1699d5a0-20bd-4b4f-8d0f-e7a4dae33289'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:43.640161 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-1699d5a0-20bd-4b4f-8d0f-e7a4dae33289 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25e4f8cfa1254b7da37d813deef79e58', 'request_id': 'req-1699d5a0-20bd-4b4f-8d0f-e7a4dae33289'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:43.649854 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-1699d5a0-20bd-4b4f-8d0f-e7a4dae33289 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25e4f8cfa1254b7da37d813deef79e58', 'request_id': 'req-1699d5a0-20bd-4b4f-8d0f-e7a4dae33289'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:43.682343 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-1699d5a0-20bd-4b4f-8d0f-e7a4dae33289 admin admin] The token cache is being invalidate because project 25e4f8cfa1254b7da37d813deef79e58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86933) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 13:46:43.697454 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-1699d5a0-20bd-4b4f-8d0f-e7a4dae33289 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:43.698995 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-1699d5a0-20bd-4b4f-8d0f-e7a4dae33289 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:43.702415 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 912/1810] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 13:46:43 2022] DELETE /identity/v3/projects/25e4f8cfa1254b7da37d813deef79e58 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:46:43.706698 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1699d5a0-20bd-4b4f-8d0f-e7a4dae33289 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:43.709032 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-1699d5a0-20bd-4b4f-8d0f-e7a4dae33289 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:43.740433 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58efe06-4caf-4a86-adec-316706bde0c2 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:43.877848 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:43.885518 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1699d5a0-20bd-4b4f-8d0f-e7a4dae33289 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:43.901738 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0267436a-3b3d-46f8-99c0-7503209c550d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:43.903251 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0267436a-3b3d-46f8-99c0-7503209c550d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:43.903640 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0267436a-3b3d-46f8-99c0-7503209c550d admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:43.904023 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0267436a-3b3d-46f8-99c0-7503209c550d admin admin] PATH_INFO: `/v3/users/ad97ae3435234e57abc371244a8758b1` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:43.909890 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0267436a-3b3d-46f8-99c0-7503209c550d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad97ae3435234e57abc371244a8758b1)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:43.912341 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0267436a-3b3d-46f8-99c0-7503209c550d admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:43.967646 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35060d4b-4bb3-427e-9f45-26afbf2eabcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:43.968295 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35060d4b-4bb3-427e-9f45-26afbf2eabcc admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:43.968966 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35060d4b-4bb3-427e-9f45-26afbf2eabcc admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:43.968966 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35060d4b-4bb3-427e-9f45-26afbf2eabcc admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:43.969629 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35060d4b-4bb3-427e-9f45-26afbf2eabcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:43.971588 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35060d4b-4bb3-427e-9f45-26afbf2eabcc admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:44.012100 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 899/1811] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:46:43 2022] GET /identity/v3/roles => generated 1420 bytes in 398 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:46:44.023915 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-0267436a-3b3d-46f8-99c0-7503209c550d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad97ae3435234e57abc371244a8758b1'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:44.024559 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35060d4b-4bb3-427e-9f45-26afbf2eabcc admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:44.029333 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-35060d4b-4bb3-427e-9f45-26afbf2eabcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:44.048650 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-0267436a-3b3d-46f8-99c0-7503209c550d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad97ae3435234e57abc371244a8758b1'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:44.068799 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-0267436a-3b3d-46f8-99c0-7503209c550d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad97ae3435234e57abc371244a8758b1'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:44.069844 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-0267436a-3b3d-46f8-99c0-7503209c550d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad97ae3435234e57abc371244a8758b1'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:44.072503 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35060d4b-4bb3-427e-9f45-26afbf2eabcc admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:44.077231 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-0267436a-3b3d-46f8-99c0-7503209c550d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad97ae3435234e57abc371244a8758b1'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:44.079681 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79ab320-98ab-4749-9166-4512b4be4c7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:44.081707 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79ab320-98ab-4749-9166-4512b4be4c7f admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:44.082503 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79ab320-98ab-4749-9166-4512b4be4c7f admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:44.082791 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79ab320-98ab-4749-9166-4512b4be4c7f admin admin] PATH_INFO: `/v3/projects/64d5eff0ff75416dbd843d51ee3ecc3c/users/124b6809816b4653ae15e3f6711813e4/roles/6e0201a9a6484f7d8b112b1eafff70bd` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:44.088263 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79ab320-98ab-4749-9166-4512b4be4c7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=64d5eff0ff75416dbd843d51ee3ecc3c, user_id=124b6809816b4653ae15e3f6711813e4, role_id=6e0201a9a6484f7d8b112b1eafff70bd)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:44.096231 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79ab320-98ab-4749-9166-4512b4be4c7f admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:44.121451 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:44.144618 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 900/1812] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 13:46:44 2022] PUT /identity/v3/projects/64d5eff0ff75416dbd843d51ee3ecc3c/users/124b6809816b4653ae15e3f6711813e4/roles/6e0201a9a6484f7d8b112b1eafff70bd => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:46:44.146578 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d8ea60-7100-44bb-aaa9-6f2879da4cb0 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:44.147029 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d8ea60-7100-44bb-aaa9-6f2879da4cb0 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:44.147320 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d8ea60-7100-44bb-aaa9-6f2879da4cb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:44.148189 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 913/1813] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 13:46:43 2022] DELETE /identity/v3/users/ad97ae3435234e57abc371244a8758b1 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 13:46:44.231693 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-e2d8ea60-7100-44bb-aaa9-6f2879da4cb0 None None] MFA Rules not processed for user `124b6809816b4653ae15e3f6711813e4`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:44.242060 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-e2d8ea60-7100-44bb-aaa9-6f2879da4cb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:44.459216 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 901/1814] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:46:44 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:46:44.471518 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0267436a-3b3d-46f8-99c0-7503209c550d admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:44.474091 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-0267436a-3b3d-46f8-99c0-7503209c550d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:44.665848 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0267436a-3b3d-46f8-99c0-7503209c550d admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:44.669743 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68b2f7c-42f8-4aeb-a25a-e055b814a727 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:44.670511 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68b2f7c-42f8-4aeb-a25a-e055b814a727 service cinder] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:44.670658 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68b2f7c-42f8-4aeb-a25a-e055b814a727 service cinder] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:44.670946 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68b2f7c-42f8-4aeb-a25a-e055b814a727 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:44.682763 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68b2f7c-42f8-4aeb-a25a-e055b814a727 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:44.684368 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68b2f7c-42f8-4aeb-a25a-e055b814a727 service cinder] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:44.725192 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 914/1815] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:46:44 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:46:45.042981 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d8ea60-7100-44bb-aaa9-6f2879da4cb0 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:45.047912 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-e2d8ea60-7100-44bb-aaa9-6f2879da4cb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:45.134265 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68b2f7c-42f8-4aeb-a25a-e055b814a727 service cinder] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:45.136394 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-c68b2f7c-42f8-4aeb-a25a-e055b814a727 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:45.205224 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d8ea60-7100-44bb-aaa9-6f2879da4cb0 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:45.221158 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30bc96a-ad8f-444b-bb3e-344586f1a8c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:45.222528 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30bc96a-ad8f-444b-bb3e-344586f1a8c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:45.223398 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30bc96a-ad8f-444b-bb3e-344586f1a8c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:45.223685 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30bc96a-ad8f-444b-bb3e-344586f1a8c6 admin admin] PATH_INFO: `/v3/users/6128975041594653b5e56cb99ab5bace` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:45.229543 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30bc96a-ad8f-444b-bb3e-344586f1a8c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6128975041594653b5e56cb99ab5bace)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:45.232631 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30bc96a-ad8f-444b-bb3e-344586f1a8c6 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:45.362954 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68b2f7c-42f8-4aeb-a25a-e055b814a727 service cinder] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:45.371763 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-d30bc96a-ad8f-444b-bb3e-344586f1a8c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6128975041594653b5e56cb99ab5bace'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:45.372435 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-d30bc96a-ad8f-444b-bb3e-344586f1a8c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6128975041594653b5e56cb99ab5bace'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:45.374121 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfdd440-36a8-448b-bb8d-5d96dc0441da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:45.376670 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfdd440-36a8-448b-bb8d-5d96dc0441da admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:45.376967 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfdd440-36a8-448b-bb8d-5d96dc0441da admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:45.377245 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfdd440-36a8-448b-bb8d-5d96dc0441da admin admin] PATH_INFO: `/v3/projects/63c5b644c2d646478c672a5d7cf6bcc2` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:45.378904 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfdd440-36a8-448b-bb8d-5d96dc0441da admin admin] RBAC: Authorizing `identity:delete_project(project_id=63c5b644c2d646478c672a5d7cf6bcc2)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:45.381258 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfdd440-36a8-448b-bb8d-5d96dc0441da admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:45.388987 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-d30bc96a-ad8f-444b-bb3e-344586f1a8c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6128975041594653b5e56cb99ab5bace'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:45.391399 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-d30bc96a-ad8f-444b-bb3e-344586f1a8c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6128975041594653b5e56cb99ab5bace'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:45.421082 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:45.430404 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-d30bc96a-ad8f-444b-bb3e-344586f1a8c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6128975041594653b5e56cb99ab5bace'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:45.454982 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:45.460855 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 902/1816] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 13:46:45 2022] DELETE /identity/v3/users/6128975041594653b5e56cb99ab5bace => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:46:45.466555 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f96b9f-4e3b-4146-a0c2-a706d2d8be47 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:45.467048 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f96b9f-4e3b-4146-a0c2-a706d2d8be47 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:45.467484 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f96b9f-4e3b-4146-a0c2-a706d2d8be47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:45.487094 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-7dfdd440-36a8-448b-bb8d-5d96dc0441da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63c5b644c2d646478c672a5d7cf6bcc2', 'request_id': 'req-7dfdd440-36a8-448b-bb8d-5d96dc0441da'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:45.498590 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-7dfdd440-36a8-448b-bb8d-5d96dc0441da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63c5b644c2d646478c672a5d7cf6bcc2', 'request_id': 'req-7dfdd440-36a8-448b-bb8d-5d96dc0441da'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:45.506798 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-7dfdd440-36a8-448b-bb8d-5d96dc0441da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63c5b644c2d646478c672a5d7cf6bcc2', 'request_id': 'req-7dfdd440-36a8-448b-bb8d-5d96dc0441da'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:45.512455 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-a8f96b9f-4e3b-4146-a0c2-a706d2d8be47 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:45.519214 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-a8f96b9f-4e3b-4146-a0c2-a706d2d8be47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:45.556189 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-7dfdd440-36a8-448b-bb8d-5d96dc0441da admin admin] The token cache is being invalidate because project 63c5b644c2d646478c672a5d7cf6bcc2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86933) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 13:46:45.557495 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-7dfdd440-36a8-448b-bb8d-5d96dc0441da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:45.560113 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-7dfdd440-36a8-448b-bb8d-5d96dc0441da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:45.571662 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 915/1817] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 13:46:45 2022] DELETE /identity/v3/projects/63c5b644c2d646478c672a5d7cf6bcc2 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:46:45.578072 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de64de66-fbc6-4282-82a4-b500981c1ed7 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:45.578413 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de64de66-fbc6-4282-82a4-b500981c1ed7 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:45.582783 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de64de66-fbc6-4282-82a4-b500981c1ed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:45.634239 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-de64de66-fbc6-4282-82a4-b500981c1ed7 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:45.635802 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-de64de66-fbc6-4282-82a4-b500981c1ed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:45.649836 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 903/1818] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:46:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 13:46:45.656806 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f96b9f-4e3b-4146-a0c2-a706d2d8be47 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:45.658835 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-a8f96b9f-4e3b-4146-a0c2-a706d2d8be47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:45.670072 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 916/1819] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:46:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:46:45.673118 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de64de66-fbc6-4282-82a4-b500981c1ed7 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:45.678515 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-de64de66-fbc6-4282-82a4-b500981c1ed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:45.699964 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f96b9f-4e3b-4146-a0c2-a706d2d8be47 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:45.724679 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280e875f-6246-45ea-85f5-69f75af4ccdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:45.725367 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280e875f-6246-45ea-85f5-69f75af4ccdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:45.725682 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280e875f-6246-45ea-85f5-69f75af4ccdd admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:45.726791 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280e875f-6246-45ea-85f5-69f75af4ccdd admin admin] PATH_INFO: `/v3/projects/df0f039f83384f83ae5d6f9e96d168ff` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:45.730983 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280e875f-6246-45ea-85f5-69f75af4ccdd admin admin] RBAC: Authorizing `identity:delete_project(project_id=df0f039f83384f83ae5d6f9e96d168ff)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:45.741456 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280e875f-6246-45ea-85f5-69f75af4ccdd admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:45.896434 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-280e875f-6246-45ea-85f5-69f75af4ccdd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df0f039f83384f83ae5d6f9e96d168ff', 'request_id': 'req-280e875f-6246-45ea-85f5-69f75af4ccdd'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:45.920713 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-280e875f-6246-45ea-85f5-69f75af4ccdd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df0f039f83384f83ae5d6f9e96d168ff', 'request_id': 'req-280e875f-6246-45ea-85f5-69f75af4ccdd'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:45.944898 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-280e875f-6246-45ea-85f5-69f75af4ccdd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df0f039f83384f83ae5d6f9e96d168ff', 'request_id': 'req-280e875f-6246-45ea-85f5-69f75af4ccdd'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:45.956898 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de64de66-fbc6-4282-82a4-b500981c1ed7 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:45.989014 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-280e875f-6246-45ea-85f5-69f75af4ccdd admin admin] The token cache is being invalidate because project df0f039f83384f83ae5d6f9e96d168ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 13:46:45.989318 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-280e875f-6246-45ea-85f5-69f75af4ccdd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:45.994715 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-280e875f-6246-45ea-85f5-69f75af4ccdd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:45.998388 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 904/1820] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 13:46:45 2022] DELETE /identity/v3/projects/df0f039f83384f83ae5d6f9e96d168ff => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:46:46.005896 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dc3d67-b4e8-4c04-97d0-8f4a115c4dce None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:46.006189 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dc3d67-b4e8-4c04-97d0-8f4a115c4dce None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:46.006467 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dc3d67-b4e8-4c04-97d0-8f4a115c4dce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:46.102133 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-a9dc3d67-b4e8-4c04-97d0-8f4a115c4dce None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:46.109008 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-a9dc3d67-b4e8-4c04-97d0-8f4a115c4dce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:46.148641 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473b3461-a881-41ee-97c4-c8c35d9592f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:46.149308 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473b3461-a881-41ee-97c4-c8c35d9592f9 service neutron] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:46.149631 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473b3461-a881-41ee-97c4-c8c35d9592f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:46.149944 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473b3461-a881-41ee-97c4-c8c35d9592f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:46.159027 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-473b3461-a881-41ee-97c4-c8c35d9592f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:46.472102 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 905/1821] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:46:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 472 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:46:46.481880 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c7c479-9af7-4aaa-b2ff-587ad47c9fc4 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:46.488470 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c7c479-9af7-4aaa-b2ff-587ad47c9fc4 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:46.488970 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c7c479-9af7-4aaa-b2ff-587ad47c9fc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:46.549319 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473b3461-a881-41ee-97c4-c8c35d9592f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:46.550941 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473b3461-a881-41ee-97c4-c8c35d9592f9 service neutron] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:46.611286 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-83c7c479-9af7-4aaa-b2ff-587ad47c9fc4 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:46.613929 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 917/1822] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:46:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 943 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:46:46.616311 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07c4606-629e-4f81-9040-6f95a7fc3b8f None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:46.621649 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07c4606-629e-4f81-9040-6f95a7fc3b8f None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:46.622483 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-83c7c479-9af7-4aaa-b2ff-587ad47c9fc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:46.623714 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07c4606-629e-4f81-9040-6f95a7fc3b8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:46.652993 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 906/1823] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:46:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:46:46.654419 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c7c479-9af7-4aaa-b2ff-587ad47c9fc4 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:46.684660 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c7c479-9af7-4aaa-b2ff-587ad47c9fc4 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:46.686976 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-c07c4606-629e-4f81-9040-6f95a7fc3b8f None None] MFA Rules not processed for user `3223a78866d94c2ba66402911c2dfae9`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:46.689493 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-c07c4606-629e-4f81-9040-6f95a7fc3b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:46.705114 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab3d7e9-c039-49fb-9c5f-732f085ce3d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:46.705812 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab3d7e9-c039-49fb-9c5f-732f085ce3d2 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:46.706119 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab3d7e9-c039-49fb-9c5f-732f085ce3d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:46.706396 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab3d7e9-c039-49fb-9c5f-732f085ce3d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:46.712905 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab3d7e9-c039-49fb-9c5f-732f085ce3d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:46:46.713282 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab3d7e9-c039-49fb-9c5f-732f085ce3d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:46.715823 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab3d7e9-c039-49fb-9c5f-732f085ce3d2 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:46.721659 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 918/1824] 10.222.0.27 () {58 vars in 1015 bytes} [Tue Jul 26 13:46:46 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:46:46.723388 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aafb83-fc92-412a-b9d2-f6e753e98320 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:46.723685 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aafb83-fc92-412a-b9d2-f6e753e98320 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:46.723981 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aafb83-fc92-412a-b9d2-f6e753e98320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:46.752657 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:46.775093 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 907/1825] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Jul 26 13:46:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:46:46.778703 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab3d7e9-c039-49fb-9c5f-732f085ce3d2 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:46.800925 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:46.801847 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-74aafb83-fc92-412a-b9d2-f6e753e98320 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:46.806326 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-74aafb83-fc92-412a-b9d2-f6e753e98320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:46.817378 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab3d7e9-c039-49fb-9c5f-732f085ce3d2 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:46.837195 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5519c7c-9eb9-4f4e-89b9-a5c43464b796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:46.838888 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 919/1826] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:46:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:46:46.841374 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5519c7c-9eb9-4f4e-89b9-a5c43464b796 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:46.841866 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5519c7c-9eb9-4f4e-89b9-a5c43464b796 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:46.842459 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5519c7c-9eb9-4f4e-89b9-a5c43464b796 admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:46.845102 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5519c7c-9eb9-4f4e-89b9-a5c43464b796 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:46.847612 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74aafb83-fc92-412a-b9d2-f6e753e98320 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:46.850888 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5519c7c-9eb9-4f4e-89b9-a5c43464b796 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:46.878703 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74aafb83-fc92-412a-b9d2-f6e753e98320 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:46.906619 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6edadd0-c951-40f6-9212-eacc26fb5adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:46.907620 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6edadd0-c951-40f6-9212-eacc26fb5adb admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:46.910505 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6edadd0-c951-40f6-9212-eacc26fb5adb admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:46.910795 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6edadd0-c951-40f6-9212-eacc26fb5adb admin admin] PATH_INFO: `/v3/domains` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:46.913939 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6edadd0-c951-40f6-9212-eacc26fb5adb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:46:46.914783 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6edadd0-c951-40f6-9212-eacc26fb5adb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:46.920403 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6edadd0-c951-40f6-9212-eacc26fb5adb admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:46.944596 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 908/1827] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 13:46:46 2022] POST /identity/v3/projects => generated 369 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:46:46.946962 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5519c7c-9eb9-4f4e-89b9-a5c43464b796 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:46.962135 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 920/1828] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Jul 26 13:46:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:46:46.963279 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6edadd0-c951-40f6-9212-eacc26fb5adb admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:46.978580 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6edadd0-c951-40f6-9212-eacc26fb5adb admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:47.193770 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eac62be-d2a5-4728-84f7-eae90574f62b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:47.194457 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eac62be-d2a5-4728-84f7-eae90574f62b admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:47.194746 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eac62be-d2a5-4728-84f7-eae90574f62b admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:47.195029 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eac62be-d2a5-4728-84f7-eae90574f62b admin admin] PATH_INFO: `/v3/projects` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:47.195741 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eac62be-d2a5-4728-84f7-eae90574f62b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:47.201704 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eac62be-d2a5-4728-84f7-eae90574f62b admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:47.254197 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5519c7c-9eb9-4f4e-89b9-a5c43464b796 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:47.287349 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56966662-9288-4081-b0f9-1f54c344ff62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:47.288839 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56966662-9288-4081-b0f9-1f54c344ff62 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:47.293094 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56966662-9288-4081-b0f9-1f54c344ff62 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:47.293391 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56966662-9288-4081-b0f9-1f54c344ff62 admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:47.297189 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56966662-9288-4081-b0f9-1f54c344ff62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:47.311631 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56966662-9288-4081-b0f9-1f54c344ff62 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:47.324321 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 921/1829] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 13:46:46 2022] POST /identity/v3/projects => generated 387 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:46:47.326016 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eac62be-d2a5-4728-84f7-eae90574f62b admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:47.345545 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-8eac62be-d2a5-4728-84f7-eae90574f62b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:47.425592 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 909/1830] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 13:46:46 2022] POST /identity/v3/users => generated 285 bytes in 480 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:46:47.427418 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56966662-9288-4081-b0f9-1f54c344ff62 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:47.494491 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eac62be-d2a5-4728-84f7-eae90574f62b admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:47.503343 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf7b64a-9e3a-4244-ba48-d82574786177 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:47.503903 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf7b64a-9e3a-4244-ba48-d82574786177 service neutron] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:47.504156 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf7b64a-9e3a-4244-ba48-d82574786177 service neutron] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:47.504491 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf7b64a-9e3a-4244-ba48-d82574786177 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:47.527031 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf7b64a-9e3a-4244-ba48-d82574786177 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:47.529914 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf7b64a-9e3a-4244-ba48-d82574786177 service neutron] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:47.532435 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56966662-9288-4081-b0f9-1f54c344ff62 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:47.553949 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 922/1831] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:46:47 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:46:47.554756 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf7b64a-9e3a-4244-ba48-d82574786177 service neutron] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:47.820928 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8193b33b-a5b1-4d87-a9bb-7e7247bb97d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:47.821756 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf7b64a-9e3a-4244-ba48-d82574786177 service neutron] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:47.826591 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193b33b-a5b1-4d87-a9bb-7e7247bb97d9 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:47.828259 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193b33b-a5b1-4d87-a9bb-7e7247bb97d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:47.830281 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193b33b-a5b1-4d87-a9bb-7e7247bb97d9 admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:47.832293 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8193b33b-a5b1-4d87-a9bb-7e7247bb97d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:47.838627 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8193b33b-a5b1-4d87-a9bb-7e7247bb97d9 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:47.848204 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ced623-8464-4dc7-a4c2-cae462663c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:47.858056 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ced623-8464-4dc7-a4c2-cae462663c73 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:47.858364 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ced623-8464-4dc7-a4c2-cae462663c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:47.860648 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ced623-8464-4dc7-a4c2-cae462663c73 admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:47.862722 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ced623-8464-4dc7-a4c2-cae462663c73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:47.868420 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ced623-8464-4dc7-a4c2-cae462663c73 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:47.904527 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 923/1832] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:46:47 2022] GET /identity/v3/roles => generated 1420 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:46:47.910386 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ced623-8464-4dc7-a4c2-cae462663c73 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:47.923348 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 910/1833] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 13:46:47 2022] POST /identity/v3/users => generated 294 bytes in 497 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:46:47.925660 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8193b33b-a5b1-4d87-a9bb-7e7247bb97d9 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:47.954459 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8193b33b-a5b1-4d87-a9bb-7e7247bb97d9 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:47.961035 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed983ee9-ad6f-468a-8ebf-5d720f0db6f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:47.961945 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed983ee9-ad6f-468a-8ebf-5d720f0db6f0 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:47.962366 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed983ee9-ad6f-468a-8ebf-5d720f0db6f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:47.962742 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed983ee9-ad6f-468a-8ebf-5d720f0db6f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:47.962848 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ced623-8464-4dc7-a4c2-cae462663c73 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:47.964145 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed983ee9-ad6f-468a-8ebf-5d720f0db6f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:47.966191 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed983ee9-ad6f-468a-8ebf-5d720f0db6f0 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:47.981789 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2af9d3c-e531-4397-9a9c-d5d2e22c73d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:47.984441 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2af9d3c-e531-4397-9a9c-d5d2e22c73d7 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:47.984441 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2af9d3c-e531-4397-9a9c-d5d2e22c73d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:47.984441 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2af9d3c-e531-4397-9a9c-d5d2e22c73d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:47.984441 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2af9d3c-e531-4397-9a9c-d5d2e22c73d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:48.002853 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 911/1834] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:46:47 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:46:48.004992 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2af9d3c-e531-4397-9a9c-d5d2e22c73d7 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:48.014785 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed983ee9-ad6f-468a-8ebf-5d720f0db6f0 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:48.023471 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:48.066689 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed983ee9-ad6f-468a-8ebf-5d720f0db6f0 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:48.076232 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 924/1835] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:46:47 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:46:48.078332 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2af9d3c-e531-4397-9a9c-d5d2e22c73d7 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:48.088535 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225b8c67-1bfc-49df-aa0f-1573886e4e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:48.090333 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225b8c67-1bfc-49df-aa0f-1573886e4e52 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:48.090333 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225b8c67-1bfc-49df-aa0f-1573886e4e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:48.093735 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225b8c67-1bfc-49df-aa0f-1573886e4e52 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:48.095205 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225b8c67-1bfc-49df-aa0f-1573886e4e52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:48.106623 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225b8c67-1bfc-49df-aa0f-1573886e4e52 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:48.145155 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2af9d3c-e531-4397-9a9c-d5d2e22c73d7 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:48.174038 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 912/1836] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:46:48 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:46:48.175409 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225b8c67-1bfc-49df-aa0f-1573886e4e52 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:48.199255 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7e7d68-53e5-4587-b9ce-4b00131d590d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:48.206307 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7e7d68-53e5-4587-b9ce-4b00131d590d admin admin] REQUEST_METHOD: `PUT` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:48.206564 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7e7d68-53e5-4587-b9ce-4b00131d590d admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:48.208011 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7e7d68-53e5-4587-b9ce-4b00131d590d admin admin] PATH_INFO: `/v3/projects/78ec27f47dec4361bec73ff96578ab39/users/e1852137064c41a8af6ba370eea4e35b/roles/759f389b221049c4b01faf3b984cd8d7` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:48.225729 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225b8c67-1bfc-49df-aa0f-1573886e4e52 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:48.244513 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5ef38d-2281-4737-8d91-1df98dabf606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:48.248239 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5ef38d-2281-4737-8d91-1df98dabf606 admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:48.250476 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5ef38d-2281-4737-8d91-1df98dabf606 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:48.251589 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5ef38d-2281-4737-8d91-1df98dabf606 admin admin] PATH_INFO: `/v3/projects/cccdac9e463a48bb8768199cf717f812/users/b32a12f59c704bec923bab59fcb8aa83/roles/759f389b221049c4b01faf3b984cd8d7` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:48.279723 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7e7d68-53e5-4587-b9ce-4b00131d590d admin admin] RBAC: Authorizing `identity:create_grant(project_id=78ec27f47dec4361bec73ff96578ab39, user_id=e1852137064c41a8af6ba370eea4e35b, role_id=759f389b221049c4b01faf3b984cd8d7)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:48.287565 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7e7d68-53e5-4587-b9ce-4b00131d590d admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:48.321078 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 925/1837] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 13:46:48 2022] PUT /identity/v3/projects/78ec27f47dec4361bec73ff96578ab39/users/e1852137064c41a8af6ba370eea4e35b/roles/759f389b221049c4b01faf3b984cd8d7 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:46:48.336666 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5ef38d-2281-4737-8d91-1df98dabf606 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cccdac9e463a48bb8768199cf717f812, user_id=b32a12f59c704bec923bab59fcb8aa83, role_id=759f389b221049c4b01faf3b984cd8d7)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:48.337349 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0223b836-8e61-4665-8ce4-5234809a75af None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:48.337698 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0223b836-8e61-4665-8ce4-5234809a75af None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:48.337981 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0223b836-8e61-4665-8ce4-5234809a75af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:48.345464 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5ef38d-2281-4737-8d91-1df98dabf606 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:48.429239 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 913/1838] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 13:46:48 2022] PUT /identity/v3/projects/cccdac9e463a48bb8768199cf717f812/users/b32a12f59c704bec923bab59fcb8aa83/roles/759f389b221049c4b01faf3b984cd8d7 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:46:48.436117 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08efbf33-d8f8-43e4-988c-dacf7ecf9958 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:48.436575 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08efbf33-d8f8-43e4-988c-dacf7ecf9958 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:48.437027 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08efbf33-d8f8-43e4-988c-dacf7ecf9958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:48.461669 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-0223b836-8e61-4665-8ce4-5234809a75af None None] MFA Rules not processed for user `e1852137064c41a8af6ba370eea4e35b`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:48.464650 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-0223b836-8e61-4665-8ce4-5234809a75af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:48.526551 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-08efbf33-d8f8-43e4-988c-dacf7ecf9958 None None] MFA Rules not processed for user `b32a12f59c704bec923bab59fcb8aa83`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:48.536063 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-08efbf33-d8f8-43e4-988c-dacf7ecf9958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:48.555404 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:48.683466 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 926/1839] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:46:48 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:46:48.683466 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0223b836-8e61-4665-8ce4-5234809a75af None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:48.684124 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-0223b836-8e61-4665-8ce4-5234809a75af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:48.771499 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 914/1840] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:46:48 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:46:48.772439 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08efbf33-d8f8-43e4-988c-dacf7ecf9958 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:48.863650 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0223b836-8e61-4665-8ce4-5234809a75af None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:48.882361 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3494163d-22cb-4cb3-98c4-6d46d78eae6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:48.884215 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3494163d-22cb-4cb3-98c4-6d46d78eae6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:48.884504 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3494163d-22cb-4cb3-98c4-6d46d78eae6a admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:48.884782 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3494163d-22cb-4cb3-98c4-6d46d78eae6a admin admin] PATH_INFO: `/v3/projects/0979dcd11d394a1d8efa7c9d6dfd3e03` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:48.892148 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3494163d-22cb-4cb3-98c4-6d46d78eae6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=0979dcd11d394a1d8efa7c9d6dfd3e03)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:48.899637 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3494163d-22cb-4cb3-98c4-6d46d78eae6a admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:49.015735 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08efbf33-d8f8-43e4-988c-dacf7ecf9958 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:49.081611 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-3494163d-22cb-4cb3-98c4-6d46d78eae6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0979dcd11d394a1d8efa7c9d6dfd3e03', 'request_id': 'req-3494163d-22cb-4cb3-98c4-6d46d78eae6a'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:49.100755 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-3494163d-22cb-4cb3-98c4-6d46d78eae6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0979dcd11d394a1d8efa7c9d6dfd3e03', 'request_id': 'req-3494163d-22cb-4cb3-98c4-6d46d78eae6a'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:49.118389 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:49.121084 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-3494163d-22cb-4cb3-98c4-6d46d78eae6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0979dcd11d394a1d8efa7c9d6dfd3e03', 'request_id': 'req-3494163d-22cb-4cb3-98c4-6d46d78eae6a'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:49.167086 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-3494163d-22cb-4cb3-98c4-6d46d78eae6a admin admin] The token cache is being invalidate because project 0979dcd11d394a1d8efa7c9d6dfd3e03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86933) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 13:46:49.168438 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-3494163d-22cb-4cb3-98c4-6d46d78eae6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:49.172256 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-3494163d-22cb-4cb3-98c4-6d46d78eae6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:49.184907 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 927/1841] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 13:46:48 2022] DELETE /identity/v3/projects/0979dcd11d394a1d8efa7c9d6dfd3e03 => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:46:49.189189 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3494163d-22cb-4cb3-98c4-6d46d78eae6a admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:49.193691 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-3494163d-22cb-4cb3-98c4-6d46d78eae6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:49.285431 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ef085a-0b3f-4b7b-b16f-795f60bc8bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:49.287822 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ef085a-0b3f-4b7b-b16f-795f60bc8bc1 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:49.288113 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ef085a-0b3f-4b7b-b16f-795f60bc8bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:49.288395 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ef085a-0b3f-4b7b-b16f-795f60bc8bc1 admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:49.291092 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ef085a-0b3f-4b7b-b16f-795f60bc8bc1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:49.297298 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ef085a-0b3f-4b7b-b16f-795f60bc8bc1 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:49.436242 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 915/1842] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 13:46:48 2022] POST /identity/v3/projects => generated 369 bytes in 664 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:46:49.439166 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ef085a-0b3f-4b7b-b16f-795f60bc8bc1 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:49.443454 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-c2ef085a-0b3f-4b7b-b16f-795f60bc8bc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:49.477402 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3494163d-22cb-4cb3-98c4-6d46d78eae6a admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:49.529331 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:49.576916 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ef085a-0b3f-4b7b-b16f-795f60bc8bc1 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:49.596723 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ece435a-31e9-4570-b4c8-054287819905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:49.598486 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ece435a-31e9-4570-b4c8-054287819905 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:49.599469 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ece435a-31e9-4570-b4c8-054287819905 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:49.600051 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ece435a-31e9-4570-b4c8-054287819905 admin admin] PATH_INFO: `/v3/users/124b6809816b4653ae15e3f6711813e4` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:49.604353 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ece435a-31e9-4570-b4c8-054287819905 admin admin] RBAC: Authorizing `identity:delete_user(user_id=124b6809816b4653ae15e3f6711813e4)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:49.611032 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ece435a-31e9-4570-b4c8-054287819905 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:49.679307 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bd78ab-6a1b-4397-8a1e-862ca50a3aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:49.680376 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bd78ab-6a1b-4397-8a1e-862ca50a3aaf admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:49.681902 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bd78ab-6a1b-4397-8a1e-862ca50a3aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:49.687456 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bd78ab-6a1b-4397-8a1e-862ca50a3aaf admin admin] PATH_INFO: `/v3/projects` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:49.691334 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bd78ab-6a1b-4397-8a1e-862ca50a3aaf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:49.698142 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bd78ab-6a1b-4397-8a1e-862ca50a3aaf admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:49.794108 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-2ece435a-31e9-4570-b4c8-054287819905 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '124b6809816b4653ae15e3f6711813e4'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:49.797270 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-2ece435a-31e9-4570-b4c8-054287819905 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '124b6809816b4653ae15e3f6711813e4'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:49.802725 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 928/1843] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 13:46:49 2022] POST /identity/v3/projects => generated 387 bytes in 617 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:46:49.803903 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bd78ab-6a1b-4397-8a1e-862ca50a3aaf admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:49.811313 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-80bd78ab-6a1b-4397-8a1e-862ca50a3aaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:49.824931 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-2ece435a-31e9-4570-b4c8-054287819905 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '124b6809816b4653ae15e3f6711813e4'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:49.826932 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-2ece435a-31e9-4570-b4c8-054287819905 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '124b6809816b4653ae15e3f6711813e4'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:49.894375 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-2ece435a-31e9-4570-b4c8-054287819905 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '124b6809816b4653ae15e3f6711813e4'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:49.939679 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 916/1844] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 13:46:49 2022] DELETE /identity/v3/users/124b6809816b4653ae15e3f6711813e4 => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:46:49.949575 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ece435a-31e9-4570-b4c8-054287819905 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:49.960234 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-2ece435a-31e9-4570-b4c8-054287819905 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:50.098626 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bd78ab-6a1b-4397-8a1e-862ca50a3aaf admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:50.173455 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ece435a-31e9-4570-b4c8-054287819905 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:50.177509 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:50.179210 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c80314-9bda-49ef-82bf-f792e4ceddad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:50.180060 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c80314-9bda-49ef-82bf-f792e4ceddad admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:50.180622 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c80314-9bda-49ef-82bf-f792e4ceddad admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:50.181009 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c80314-9bda-49ef-82bf-f792e4ceddad admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:50.182041 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c80314-9bda-49ef-82bf-f792e4ceddad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:50.184260 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c80314-9bda-49ef-82bf-f792e4ceddad admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:50.222097 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 917/1845] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 13:46:49 2022] POST /identity/v3/users => generated 300 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:46:50.225821 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c80314-9bda-49ef-82bf-f792e4ceddad admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:50.244770 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c80314-9bda-49ef-82bf-f792e4ceddad admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:50.255006 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b70087-27a9-4873-bc59-fb1cb380bfa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:50.256590 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b70087-27a9-4873-bc59-fb1cb380bfa9 admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:50.257309 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b70087-27a9-4873-bc59-fb1cb380bfa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:50.258336 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b70087-27a9-4873-bc59-fb1cb380bfa9 admin admin] PATH_INFO: `/v3/users` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:50.260829 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b70087-27a9-4873-bc59-fb1cb380bfa9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:50.263708 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa74892-23c9-47f3-935d-46fbf45125c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:50.264465 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa74892-23c9-47f3-935d-46fbf45125c8 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:50.264926 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa74892-23c9-47f3-935d-46fbf45125c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:50.265339 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa74892-23c9-47f3-935d-46fbf45125c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:50.266366 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa74892-23c9-47f3-935d-46fbf45125c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:50.267574 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b70087-27a9-4873-bc59-fb1cb380bfa9 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:50.268824 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa74892-23c9-47f3-935d-46fbf45125c8 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:50.285389 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 918/1846] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:46:50 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:46:50.289397 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa74892-23c9-47f3-935d-46fbf45125c8 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:50.303779 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa74892-23c9-47f3-935d-46fbf45125c8 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:50.313543 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411d950a-bce7-4827-95eb-89c15c469bb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:50.315003 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411d950a-bce7-4827-95eb-89c15c469bb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:50.315401 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411d950a-bce7-4827-95eb-89c15c469bb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:50.315945 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411d950a-bce7-4827-95eb-89c15c469bb9 admin admin] PATH_INFO: `/v3/projects/18fa4c5418c8450aa27cdf1069ab8d22/users/b12c0672cab44c8ba18ee5e1f60c8659/roles/6e0201a9a6484f7d8b112b1eafff70bd` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:50.338533 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 929/1847] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 13:46:49 2022] POST /identity/v3/users => generated 291 bytes in 534 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:46:50.339531 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b70087-27a9-4873-bc59-fb1cb380bfa9 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:50.363036 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411d950a-bce7-4827-95eb-89c15c469bb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18fa4c5418c8450aa27cdf1069ab8d22, user_id=b12c0672cab44c8ba18ee5e1f60c8659, role_id=6e0201a9a6484f7d8b112b1eafff70bd)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:50.369574 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b70087-27a9-4873-bc59-fb1cb380bfa9 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:50.372536 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411d950a-bce7-4827-95eb-89c15c469bb9 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:50.383095 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eabbc61-0700-4aae-8834-83f364b5c9bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:50.383908 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eabbc61-0700-4aae-8834-83f364b5c9bc admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:50.384173 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eabbc61-0700-4aae-8834-83f364b5c9bc admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:50.384454 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eabbc61-0700-4aae-8834-83f364b5c9bc admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:50.385324 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eabbc61-0700-4aae-8834-83f364b5c9bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:50.388902 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eabbc61-0700-4aae-8834-83f364b5c9bc admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:50.424901 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 930/1848] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:46:50 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:46:50.427163 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eabbc61-0700-4aae-8834-83f364b5c9bc admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:50.431117 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 919/1849] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 13:46:50 2022] PUT /identity/v3/projects/18fa4c5418c8450aa27cdf1069ab8d22/users/b12c0672cab44c8ba18ee5e1f60c8659/roles/6e0201a9a6484f7d8b112b1eafff70bd => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:46:50.433004 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411d950a-bce7-4827-95eb-89c15c469bb9 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:50.449288 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eabbc61-0700-4aae-8834-83f364b5c9bc admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:50.482433 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=85,keystone:DELETE=10,keystone:INSERT=5 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:50.606838 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411d950a-bce7-4827-95eb-89c15c469bb9 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:50.615254 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2837fe39-e351-4d1d-b27e-c3e23f050a90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:50.616188 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2837fe39-e351-4d1d-b27e-c3e23f050a90 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:50.616613 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2837fe39-e351-4d1d-b27e-c3e23f050a90 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:50.616994 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2837fe39-e351-4d1d-b27e-c3e23f050a90 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:50.618556 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2837fe39-e351-4d1d-b27e-c3e23f050a90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:50.620702 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2837fe39-e351-4d1d-b27e-c3e23f050a90 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:50.643971 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 920/1850] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:46:50 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:46:50.645147 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2837fe39-e351-4d1d-b27e-c3e23f050a90 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:50.647323 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d9ad01-ea21-41d9-a260-44804d18bf43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:50.649277 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d9ad01-ea21-41d9-a260-44804d18bf43 admin admin] REQUEST_METHOD: `PUT` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:50.649602 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d9ad01-ea21-41d9-a260-44804d18bf43 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:50.649889 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d9ad01-ea21-41d9-a260-44804d18bf43 admin admin] PATH_INFO: `/v3/projects/4aa9fe91588547b5887bc5333a782f81/users/5bda2e864e524ad28ae9f165058c463d/roles/6e0201a9a6484f7d8b112b1eafff70bd` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:50.671818 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2837fe39-e351-4d1d-b27e-c3e23f050a90 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:50.683184 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47be10a-728d-4745-8bc3-7f269f7ed2bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:50.684683 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47be10a-728d-4745-8bc3-7f269f7ed2bb admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:50.685433 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47be10a-728d-4745-8bc3-7f269f7ed2bb admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:50.685898 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47be10a-728d-4745-8bc3-7f269f7ed2bb admin admin] PATH_INFO: `/v3/projects/18fa4c5418c8450aa27cdf1069ab8d22/users/b12c0672cab44c8ba18ee5e1f60c8659/roles/6e0201a9a6484f7d8b112b1eafff70bd` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:50.689997 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47be10a-728d-4745-8bc3-7f269f7ed2bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=18fa4c5418c8450aa27cdf1069ab8d22, user_id=b12c0672cab44c8ba18ee5e1f60c8659, role_id=6e0201a9a6484f7d8b112b1eafff70bd)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:50.697179 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47be10a-728d-4745-8bc3-7f269f7ed2bb admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:50.704012 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d9ad01-ea21-41d9-a260-44804d18bf43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4aa9fe91588547b5887bc5333a782f81, user_id=5bda2e864e524ad28ae9f165058c463d, role_id=6e0201a9a6484f7d8b112b1eafff70bd)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:50.707494 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d9ad01-ea21-41d9-a260-44804d18bf43 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:50.743814 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 931/1852] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 13:46:50 2022] PUT /identity/v3/projects/4aa9fe91588547b5887bc5333a782f81/users/5bda2e864e524ad28ae9f165058c463d/roles/6e0201a9a6484f7d8b112b1eafff70bd => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:46:50.745631 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb08e58a-d210-4688-8c4b-88a06b3a97c0 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:50.745976 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb08e58a-d210-4688-8c4b-88a06b3a97c0 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:50.746317 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb08e58a-d210-4688-8c4b-88a06b3a97c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:50.746607 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 921/1852] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 13:46:50 2022] PUT /identity/v3/projects/18fa4c5418c8450aa27cdf1069ab8d22/users/b12c0672cab44c8ba18ee5e1f60c8659/roles/6e0201a9a6484f7d8b112b1eafff70bd => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:46:50.751262 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47be10a-728d-4745-8bc3-7f269f7ed2bb admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:50.844322 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-eb08e58a-d210-4688-8c4b-88a06b3a97c0 None None] MFA Rules not processed for user `b12c0672cab44c8ba18ee5e1f60c8659`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:50.847849 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-eb08e58a-d210-4688-8c4b-88a06b3a97c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:50.977215 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47be10a-728d-4745-8bc3-7f269f7ed2bb admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:51.025532 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 932/1853] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:46:50 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:46:51.040109 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb08e58a-d210-4688-8c4b-88a06b3a97c0 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:51.041557 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-eb08e58a-d210-4688-8c4b-88a06b3a97c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:51.090575 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:51.135728 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11cbbee-6c47-45d3-a7ea-c9c9eb81e2ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:51.139929 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11cbbee-6c47-45d3-a7ea-c9c9eb81e2ef admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:51.143036 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11cbbee-6c47-45d3-a7ea-c9c9eb81e2ef admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:51.143909 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11cbbee-6c47-45d3-a7ea-c9c9eb81e2ef admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:51.146612 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11cbbee-6c47-45d3-a7ea-c9c9eb81e2ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:51.153230 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11cbbee-6c47-45d3-a7ea-c9c9eb81e2ef admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:51.192697 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 922/1854] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:46:50 2022] GET /identity/v3/roles => generated 1420 bytes in 446 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:46:51.201324 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11cbbee-6c47-45d3-a7ea-c9c9eb81e2ef admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:51.211554 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb08e58a-d210-4688-8c4b-88a06b3a97c0 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:51.226396 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11cbbee-6c47-45d3-a7ea-c9c9eb81e2ef admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:51.230418 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03826196-5afe-476e-b5f9-7bab2d41ab08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:51.231068 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03826196-5afe-476e-b5f9-7bab2d41ab08 service cinder] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:51.231353 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03826196-5afe-476e-b5f9-7bab2d41ab08 service cinder] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:51.231632 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03826196-5afe-476e-b5f9-7bab2d41ab08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:51.242072 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ceb790-a6af-4b41-bdea-b08c023446ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:51.243230 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ceb790-a6af-4b41-bdea-b08c023446ae admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:51.244485 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ceb790-a6af-4b41-bdea-b08c023446ae admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:51.245092 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ceb790-a6af-4b41-bdea-b08c023446ae admin admin] PATH_INFO: `/v3/projects/4aa9fe91588547b5887bc5333a782f81/users/5bda2e864e524ad28ae9f165058c463d/roles/6e0201a9a6484f7d8b112b1eafff70bd` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:51.248988 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ceb790-a6af-4b41-bdea-b08c023446ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=4aa9fe91588547b5887bc5333a782f81, user_id=5bda2e864e524ad28ae9f165058c463d, role_id=6e0201a9a6484f7d8b112b1eafff70bd)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:51.256142 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ceb790-a6af-4b41-bdea-b08c023446ae admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:51.284231 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 923/1855] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 13:46:51 2022] PUT /identity/v3/projects/4aa9fe91588547b5887bc5333a782f81/users/5bda2e864e524ad28ae9f165058c463d/roles/6e0201a9a6484f7d8b112b1eafff70bd => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:46:51.291482 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d95bc87-87b1-40aa-9578-715ada9b2e4e None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:51.291777 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d95bc87-87b1-40aa-9578-715ada9b2e4e None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:51.292086 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d95bc87-87b1-40aa-9578-715ada9b2e4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:51.294562 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03826196-5afe-476e-b5f9-7bab2d41ab08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:51.296029 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03826196-5afe-476e-b5f9-7bab2d41ab08 service cinder] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:51.328841 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 933/1856] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:46:51 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 13:46:51.386656 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-9d95bc87-87b1-40aa-9578-715ada9b2e4e None None] MFA Rules not processed for user `5bda2e864e524ad28ae9f165058c463d`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:51.398769 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-9d95bc87-87b1-40aa-9578-715ada9b2e4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:51.454723 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:51.471053 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03826196-5afe-476e-b5f9-7bab2d41ab08 service cinder] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:51.489024 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03826196-5afe-476e-b5f9-7bab2d41ab08 service cinder] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:51.643868 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97256390-6897-499d-ac4f-666a18bfb165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:51.646249 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97256390-6897-499d-ac4f-666a18bfb165 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:51.646721 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97256390-6897-499d-ac4f-666a18bfb165 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:51.647187 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97256390-6897-499d-ac4f-666a18bfb165 admin admin] PATH_INFO: `/v3/users/b32a12f59c704bec923bab59fcb8aa83` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:51.651282 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97256390-6897-499d-ac4f-666a18bfb165 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b32a12f59c704bec923bab59fcb8aa83)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:51.657000 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97256390-6897-499d-ac4f-666a18bfb165 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:51.756644 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 924/1857] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:46:51 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 468 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:46:51.779227 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-97256390-6897-499d-ac4f-666a18bfb165 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b32a12f59c704bec923bab59fcb8aa83'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:51.794819 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d95bc87-87b1-40aa-9578-715ada9b2e4e None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:51.795435 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-97256390-6897-499d-ac4f-666a18bfb165 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b32a12f59c704bec923bab59fcb8aa83'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:51.798629 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-9d95bc87-87b1-40aa-9578-715ada9b2e4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:51.809158 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-97256390-6897-499d-ac4f-666a18bfb165 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b32a12f59c704bec923bab59fcb8aa83'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:51.810063 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-97256390-6897-499d-ac4f-666a18bfb165 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b32a12f59c704bec923bab59fcb8aa83'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:51.811056 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-97256390-6897-499d-ac4f-666a18bfb165 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b32a12f59c704bec923bab59fcb8aa83'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:51.927691 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 934/1858] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 13:46:51 2022] DELETE /identity/v3/users/b32a12f59c704bec923bab59fcb8aa83 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:46:51.934228 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3545b765-91ae-4cba-ab87-f8ac15e1ec4e None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:51.934527 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3545b765-91ae-4cba-ab87-f8ac15e1ec4e None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:51.937373 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3545b765-91ae-4cba-ab87-f8ac15e1ec4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:51.966667 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-3545b765-91ae-4cba-ab87-f8ac15e1ec4e None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:51.968559 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-3545b765-91ae-4cba-ab87-f8ac15e1ec4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:52.016172 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:52.051521 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d95bc87-87b1-40aa-9578-715ada9b2e4e None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:52.071133 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 935/1859] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:46:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:46:52.105760 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3545b765-91ae-4cba-ab87-f8ac15e1ec4e None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:52.107975 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-3545b765-91ae-4cba-ab87-f8ac15e1ec4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:52.277834 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7085779-290e-46c8-ad89-a182ec769741 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:52.281682 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7085779-290e-46c8-ad89-a182ec769741 service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:52.283572 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7085779-290e-46c8-ad89-a182ec769741 service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:52.285748 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7085779-290e-46c8-ad89-a182ec769741 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:52.292628 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-c7085779-290e-46c8-ad89-a182ec769741 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:52.323990 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3545b765-91ae-4cba-ab87-f8ac15e1ec4e None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:52.329034 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b535a2fa-2561-4c57-bf11-3fc88a3501b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:52.330367 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b535a2fa-2561-4c57-bf11-3fc88a3501b9 service neutron] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:52.330807 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b535a2fa-2561-4c57-bf11-3fc88a3501b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:52.331306 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b535a2fa-2561-4c57-bf11-3fc88a3501b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:52.346030 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b535a2fa-2561-4c57-bf11-3fc88a3501b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:52.348229 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b535a2fa-2561-4c57-bf11-3fc88a3501b9 service neutron] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:52.362672 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 936/1860] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:46:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:46:52.363742 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b535a2fa-2561-4c57-bf11-3fc88a3501b9 service neutron] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:52.365554 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-b535a2fa-2561-4c57-bf11-3fc88a3501b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:52.381392 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b535a2fa-2561-4c57-bf11-3fc88a3501b9 service neutron] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:52.393304 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53c3699-ff76-4a84-9af3-a5865c8b7c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:52.394337 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c3699-ff76-4a84-9af3-a5865c8b7c70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:52.397201 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c3699-ff76-4a84-9af3-a5865c8b7c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:52.397656 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c3699-ff76-4a84-9af3-a5865c8b7c70 admin admin] PATH_INFO: `/v3/projects/64d5eff0ff75416dbd843d51ee3ecc3c` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:52.399127 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53c3699-ff76-4a84-9af3-a5865c8b7c70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64d5eff0ff75416dbd843d51ee3ecc3c)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:52.402927 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53c3699-ff76-4a84-9af3-a5865c8b7c70 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:52.490900 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7085779-290e-46c8-ad89-a182ec769741 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:52.492508 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7085779-290e-46c8-ad89-a182ec769741 service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:52.521332 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 925/1861] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:46:51 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 727 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:46:52.537019 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-c53c3699-ff76-4a84-9af3-a5865c8b7c70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64d5eff0ff75416dbd843d51ee3ecc3c', 'request_id': 'req-c53c3699-ff76-4a84-9af3-a5865c8b7c70'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:52.551169 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-c53c3699-ff76-4a84-9af3-a5865c8b7c70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64d5eff0ff75416dbd843d51ee3ecc3c', 'request_id': 'req-c53c3699-ff76-4a84-9af3-a5865c8b7c70'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:52.558625 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-c53c3699-ff76-4a84-9af3-a5865c8b7c70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64d5eff0ff75416dbd843d51ee3ecc3c', 'request_id': 'req-c53c3699-ff76-4a84-9af3-a5865c8b7c70'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:52.582927 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-c53c3699-ff76-4a84-9af3-a5865c8b7c70 admin admin] The token cache is being invalidate because project 64d5eff0ff75416dbd843d51ee3ecc3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86933) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 13:46:52.583278 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-c53c3699-ff76-4a84-9af3-a5865c8b7c70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:52.587197 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7085779-290e-46c8-ad89-a182ec769741 service cinder] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:52.587375 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-c53c3699-ff76-4a84-9af3-a5865c8b7c70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:52.588910 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-c7085779-290e-46c8-ad89-a182ec769741 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:52.592172 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 937/1862] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 13:46:52 2022] DELETE /identity/v3/projects/64d5eff0ff75416dbd843d51ee3ecc3c => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:46:52.598978 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5912cbe8-ee81-424a-8eac-61a138d3a7a9 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:52.598978 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5912cbe8-ee81-424a-8eac-61a138d3a7a9 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:52.598978 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5912cbe8-ee81-424a-8eac-61a138d3a7a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:52.612543 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7085779-290e-46c8-ad89-a182ec769741 service cinder] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:52.633710 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:52.679385 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-5912cbe8-ee81-424a-8eac-61a138d3a7a9 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:52.680634 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-5912cbe8-ee81-424a-8eac-61a138d3a7a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:52.697609 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9972ee7-9706-4d94-82b3-59be76f5adc6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:52.698697 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9972ee7-9706-4d94-82b3-59be76f5adc6 service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:52.699085 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9972ee7-9706-4d94-82b3-59be76f5adc6 service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:52.699465 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9972ee7-9706-4d94-82b3-59be76f5adc6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:52.701618 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-d9972ee7-9706-4d94-82b3-59be76f5adc6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:52.902474 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 938/1863] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:46:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 13:46:52.913682 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41785f07-037c-4329-b516-8caffce7287c None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:52.922808 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41785f07-037c-4329-b516-8caffce7287c None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:52.923241 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41785f07-037c-4329-b516-8caffce7287c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:52.973643 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9972ee7-9706-4d94-82b3-59be76f5adc6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:52.975190 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9972ee7-9706-4d94-82b3-59be76f5adc6 service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:53.026938 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:53.053372 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 926/1864] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:46:52 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:46:53.055018 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-41785f07-037c-4329-b516-8caffce7287c None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:53.067422 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-41785f07-037c-4329-b516-8caffce7287c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:53.110703 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 939/1865] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:46:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:46:53.112719 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9972ee7-9706-4d94-82b3-59be76f5adc6 service cinder] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:53.145794 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9972ee7-9706-4d94-82b3-59be76f5adc6 service cinder] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:53.183059 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb8ae47-fe5b-4c38-b826-c9a78e8588cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:53.183797 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb8ae47-fe5b-4c38-b826-c9a78e8588cf admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:53.184152 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb8ae47-fe5b-4c38-b826-c9a78e8588cf admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:53.184433 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb8ae47-fe5b-4c38-b826-c9a78e8588cf admin admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:53.185159 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb8ae47-fe5b-4c38-b826-c9a78e8588cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:46:53.185488 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb8ae47-fe5b-4c38-b826-c9a78e8588cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:53.220602 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb8ae47-fe5b-4c38-b826-c9a78e8588cf admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:53.305970 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41785f07-037c-4329-b516-8caffce7287c None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:53.310202 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 927/1866] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Jul 26 13:46:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:46:53.334327 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41785f07-037c-4329-b516-8caffce7287c None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:53.356656 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c98f2eb-e540-4286-82fa-f701a1ecf44d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:53.360142 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c98f2eb-e540-4286-82fa-f701a1ecf44d admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:53.362411 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c98f2eb-e540-4286-82fa-f701a1ecf44d admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:53.363145 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c98f2eb-e540-4286-82fa-f701a1ecf44d admin admin] PATH_INFO: `/v3/projects` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:53.365396 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c98f2eb-e540-4286-82fa-f701a1ecf44d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:53.370952 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c98f2eb-e540-4286-82fa-f701a1ecf44d admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:53.467930 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 940/1867] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 13:46:53 2022] POST /identity/v3/projects => generated 367 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 13:46:53.473692 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb8ae47-fe5b-4c38-b826-c9a78e8588cf admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:53.502733 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb8ae47-fe5b-4c38-b826-c9a78e8588cf admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:53.754732 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d49b73-00e1-4b62-b185-902ce9f8f50e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:53.755666 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d49b73-00e1-4b62-b185-902ce9f8f50e admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:53.755983 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d49b73-00e1-4b62-b185-902ce9f8f50e admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:53.756400 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d49b73-00e1-4b62-b185-902ce9f8f50e admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:53.757030 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d49b73-00e1-4b62-b185-902ce9f8f50e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:53.759237 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d49b73-00e1-4b62-b185-902ce9f8f50e admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:53.807279 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 928/1868] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 13:46:53 2022] POST /identity/v3/users => generated 284 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:46:53.813246 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c98f2eb-e540-4286-82fa-f701a1ecf44d admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:53.827146 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c98f2eb-e540-4286-82fa-f701a1ecf44d admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:53.853603 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdf40ca-727f-4c8c-ab31-7a997b0d0b4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:53.854334 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdf40ca-727f-4c8c-ab31-7a997b0d0b4d admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:53.854662 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdf40ca-727f-4c8c-ab31-7a997b0d0b4d admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:53.854945 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdf40ca-727f-4c8c-ab31-7a997b0d0b4d admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:53.855657 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdf40ca-727f-4c8c-ab31-7a997b0d0b4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:53.864784 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdf40ca-727f-4c8c-ab31-7a997b0d0b4d admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:53.901942 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 941/1869] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:46:53 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:46:53.904750 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d49b73-00e1-4b62-b185-902ce9f8f50e admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:53.927055 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d49b73-00e1-4b62-b185-902ce9f8f50e admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:53.947013 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaedabd5-54d8-4f34-a984-f51e9c8b10ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:53.947759 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaedabd5-54d8-4f34-a984-f51e9c8b10ca admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:53.948129 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaedabd5-54d8-4f34-a984-f51e9c8b10ca admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:53.948451 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaedabd5-54d8-4f34-a984-f51e9c8b10ca admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:53.949181 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaedabd5-54d8-4f34-a984-f51e9c8b10ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:53.951071 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaedabd5-54d8-4f34-a984-f51e9c8b10ca admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:54.004815 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 929/1870] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:46:53 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 13:46:54.007040 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdf40ca-727f-4c8c-ab31-7a997b0d0b4d admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:54.048832 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdf40ca-727f-4c8c-ab31-7a997b0d0b4d admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:54.067688 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfb6081-4f5b-443d-9983-a011e44f338e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:54.068560 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb6081-4f5b-443d-9983-a011e44f338e admin admin] REQUEST_METHOD: `PUT` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:54.068975 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb6081-4f5b-443d-9983-a011e44f338e admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:54.069563 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb6081-4f5b-443d-9983-a011e44f338e admin admin] PATH_INFO: `/v3/projects/f50e691cd38341c5a9e1251871ac4fdb/users/aed84db1720b4bb2b1d083f9a8f998f0/roles/759f389b221049c4b01faf3b984cd8d7` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:54.105655 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfb6081-4f5b-443d-9983-a011e44f338e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f50e691cd38341c5a9e1251871ac4fdb, user_id=aed84db1720b4bb2b1d083f9a8f998f0, role_id=759f389b221049c4b01faf3b984cd8d7)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:54.109184 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfb6081-4f5b-443d-9983-a011e44f338e admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:54.145951 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 942/1871] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 13:46:54 2022] PUT /identity/v3/projects/f50e691cd38341c5a9e1251871ac4fdb/users/aed84db1720b4bb2b1d083f9a8f998f0/roles/759f389b221049c4b01faf3b984cd8d7 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 13:46:54.157823 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2f1230-93ec-4fa4-ac21-59faecc22b02 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:54.159676 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477d4bf6-e57e-45b7-9b26-ec2b1b209f05 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:54.160301 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2f1230-93ec-4fa4-ac21-59faecc22b02 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:54.160851 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477d4bf6-e57e-45b7-9b26-ec2b1b209f05 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:54.161340 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2f1230-93ec-4fa4-ac21-59faecc22b02 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:54.161832 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477d4bf6-e57e-45b7-9b26-ec2b1b209f05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:54.167081 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 930/1872] 10.222.0.58 () {54 vars in 921 bytes} [Tue Jul 26 13:46:54 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:46:54.240565 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-477d4bf6-e57e-45b7-9b26-ec2b1b209f05 None None] MFA Rules not processed for user `aed84db1720b4bb2b1d083f9a8f998f0`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:54.242039 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-477d4bf6-e57e-45b7-9b26-ec2b1b209f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:54.399189 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 943/1873] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:46:54 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:46:54.403713 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2f1230-93ec-4fa4-ac21-59faecc22b02 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:54.470753 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:54.509652 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19464860-1e09-4361-98a7-a7397dac9fef None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:54.510329 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19464860-1e09-4361-98a7-a7397dac9fef None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:54.510682 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19464860-1e09-4361-98a7-a7397dac9fef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:54.558801 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2f1230-93ec-4fa4-ac21-59faecc22b02 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:54.585009 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-19464860-1e09-4361-98a7-a7397dac9fef None None] MFA Rules not processed for user `c69161cd520b4b29bf50f3bf2f1c033b`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:54.608939 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-19464860-1e09-4361-98a7-a7397dac9fef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:54.609469 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a436fd-d639-42b2-a935-8c826820a4e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:54.609469 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a436fd-d639-42b2-a935-8c826820a4e1 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:54.609469 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a436fd-d639-42b2-a935-8c826820a4e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:54.609469 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a436fd-d639-42b2-a935-8c826820a4e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:54.610272 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a436fd-d639-42b2-a935-8c826820a4e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:54.617347 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a436fd-d639-42b2-a935-8c826820a4e1 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:54.774846 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 931/1874] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 13:46:54 2022] POST /identity/v3/projects => generated 365 bytes in 365 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:46:54.779887 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a436fd-d639-42b2-a935-8c826820a4e1 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:54.856671 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 944/1875] 10.222.0.58 () {58 vars in 1018 bytes} [Tue Jul 26 13:46:54 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:46:54.857749 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19464860-1e09-4361-98a7-a7397dac9fef None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:54.859136 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-19464860-1e09-4361-98a7-a7397dac9fef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:54.889111 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19464860-1e09-4361-98a7-a7397dac9fef None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:54.896517 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1657673f-533f-43b6-afae-bb1e6041b1d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:54.897390 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1657673f-533f-43b6-afae-bb1e6041b1d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:54.898319 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1657673f-533f-43b6-afae-bb1e6041b1d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:54.898666 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1657673f-533f-43b6-afae-bb1e6041b1d8 admin admin] PATH_INFO: `/v3/projects/cccdac9e463a48bb8768199cf717f812` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:54.900307 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1657673f-533f-43b6-afae-bb1e6041b1d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cccdac9e463a48bb8768199cf717f812)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:54.902621 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a436fd-d639-42b2-a935-8c826820a4e1 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:54.903493 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1657673f-533f-43b6-afae-bb1e6041b1d8 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:54.911433 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6c685d-789b-4ec2-b480-d45d373d7f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:54.912335 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6c685d-789b-4ec2-b480-d45d373d7f1f admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:54.912780 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6c685d-789b-4ec2-b480-d45d373d7f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:54.913225 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6c685d-789b-4ec2-b480-d45d373d7f1f admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:54.914713 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6c685d-789b-4ec2-b480-d45d373d7f1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:54.918904 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6c685d-789b-4ec2-b480-d45d373d7f1f admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:54.950428 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 932/1876] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 13:46:54 2022] POST /identity/v3/users => generated 289 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 13:46:54.961046 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6c685d-789b-4ec2-b480-d45d373d7f1f admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:54.993430 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6c685d-789b-4ec2-b480-d45d373d7f1f admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:55.007325 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3303543a-ae21-435c-a473-da8af8dd4223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:55.010179 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3303543a-ae21-435c-a473-da8af8dd4223 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:55.010468 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3303543a-ae21-435c-a473-da8af8dd4223 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:55.011171 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3303543a-ae21-435c-a473-da8af8dd4223 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:55.012922 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3303543a-ae21-435c-a473-da8af8dd4223 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:55.022954 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3303543a-ae21-435c-a473-da8af8dd4223 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:55.055116 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 933/1877] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:46:54 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:46:55.058970 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3303543a-ae21-435c-a473-da8af8dd4223 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:55.063259 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:55.088106 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3303543a-ae21-435c-a473-da8af8dd4223 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:55.107902 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6cac14-283f-4e19-bf40-308afa9a52b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:55.108607 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6cac14-283f-4e19-bf40-308afa9a52b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:55.108891 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6cac14-283f-4e19-bf40-308afa9a52b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:55.109249 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6cac14-283f-4e19-bf40-308afa9a52b8 admin admin] PATH_INFO: `/v3/projects/dfd7e70158a140e9a8f257e76670f756/users/b54341f7a6ab4450b53924664cb4732a/roles/6e0201a9a6484f7d8b112b1eafff70bd` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:55.154307 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-1657673f-533f-43b6-afae-bb1e6041b1d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cccdac9e463a48bb8768199cf717f812', 'request_id': 'req-1657673f-533f-43b6-afae-bb1e6041b1d8'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:55.174569 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-1657673f-533f-43b6-afae-bb1e6041b1d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cccdac9e463a48bb8768199cf717f812', 'request_id': 'req-1657673f-533f-43b6-afae-bb1e6041b1d8'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:55.177487 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6cac14-283f-4e19-bf40-308afa9a52b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfd7e70158a140e9a8f257e76670f756, user_id=b54341f7a6ab4450b53924664cb4732a, role_id=6e0201a9a6484f7d8b112b1eafff70bd)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:55.186995 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-1657673f-533f-43b6-afae-bb1e6041b1d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cccdac9e463a48bb8768199cf717f812', 'request_id': 'req-1657673f-533f-43b6-afae-bb1e6041b1d8'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:55.192822 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6cac14-283f-4e19-bf40-308afa9a52b8 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:55.215953 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-1657673f-533f-43b6-afae-bb1e6041b1d8 admin admin] The token cache is being invalidate because project cccdac9e463a48bb8768199cf717f812 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86933) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 13:46:55.216351 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-1657673f-533f-43b6-afae-bb1e6041b1d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:55.217320 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-1657673f-533f-43b6-afae-bb1e6041b1d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:55.224288 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 945/1878] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 13:46:54 2022] DELETE /identity/v3/projects/cccdac9e463a48bb8768199cf717f812 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:46:55.225310 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1657673f-533f-43b6-afae-bb1e6041b1d8 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:55.227562 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-1657673f-533f-43b6-afae-bb1e6041b1d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:55.242499 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 934/1879] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 13:46:55 2022] PUT /identity/v3/projects/dfd7e70158a140e9a8f257e76670f756/users/b54341f7a6ab4450b53924664cb4732a/roles/6e0201a9a6484f7d8b112b1eafff70bd => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 13:46:55.246002 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6cac14-283f-4e19-bf40-308afa9a52b8 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:55.251475 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-0c6cac14-283f-4e19-bf40-308afa9a52b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:55.424871 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1657673f-533f-43b6-afae-bb1e6041b1d8 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:55.481427 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6cac14-283f-4e19-bf40-308afa9a52b8 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:55.495855 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac5941e-8ac8-4eef-a32f-1119cbb51795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:55.497853 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac5941e-8ac8-4eef-a32f-1119cbb51795 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:55.499585 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac5941e-8ac8-4eef-a32f-1119cbb51795 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:55.500999 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac5941e-8ac8-4eef-a32f-1119cbb51795 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:55.504048 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac5941e-8ac8-4eef-a32f-1119cbb51795 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:55.509986 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac5941e-8ac8-4eef-a32f-1119cbb51795 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:55.553538 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 935/1880] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:46:55 2022] GET /identity/v3/roles => generated 1420 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:46:55.554981 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac5941e-8ac8-4eef-a32f-1119cbb51795 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:55.605720 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac5941e-8ac8-4eef-a32f-1119cbb51795 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:55.624173 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24877dd-e04b-48c8-8e2c-7e73f4bd52ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:55.625117 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24877dd-e04b-48c8-8e2c-7e73f4bd52ac admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:55.625396 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24877dd-e04b-48c8-8e2c-7e73f4bd52ac admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:55.625728 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24877dd-e04b-48c8-8e2c-7e73f4bd52ac admin admin] PATH_INFO: `/v3/projects/dfd7e70158a140e9a8f257e76670f756/users/b54341f7a6ab4450b53924664cb4732a/roles/6e0201a9a6484f7d8b112b1eafff70bd` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:55.636546 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24877dd-e04b-48c8-8e2c-7e73f4bd52ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfd7e70158a140e9a8f257e76670f756, user_id=b54341f7a6ab4450b53924664cb4732a, role_id=6e0201a9a6484f7d8b112b1eafff70bd)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:55.641865 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24877dd-e04b-48c8-8e2c-7e73f4bd52ac admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:55.654928 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1966a751-a610-4e01-bb2e-43f32a15b7ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:55.655974 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1966a751-a610-4e01-bb2e-43f32a15b7ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:55.656893 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1966a751-a610-4e01-bb2e-43f32a15b7ff admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:55.659072 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1966a751-a610-4e01-bb2e-43f32a15b7ff admin admin] PATH_INFO: `/v3/users/b12c0672cab44c8ba18ee5e1f60c8659` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:55.661989 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1966a751-a610-4e01-bb2e-43f32a15b7ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=b12c0672cab44c8ba18ee5e1f60c8659)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:55.664110 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1966a751-a610-4e01-bb2e-43f32a15b7ff admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:55.667912 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:55.685565 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 936/1881] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 13:46:55 2022] PUT /identity/v3/projects/dfd7e70158a140e9a8f257e76670f756/users/b54341f7a6ab4450b53924664cb4732a/roles/6e0201a9a6484f7d8b112b1eafff70bd => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:46:55.686913 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24877dd-e04b-48c8-8e2c-7e73f4bd52ac admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:55.696988 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-a24877dd-e04b-48c8-8e2c-7e73f4bd52ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:55.835872 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-1966a751-a610-4e01-bb2e-43f32a15b7ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b12c0672cab44c8ba18ee5e1f60c8659'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:55.872753 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-1966a751-a610-4e01-bb2e-43f32a15b7ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b12c0672cab44c8ba18ee5e1f60c8659'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:55.875867 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:55.895397 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-1966a751-a610-4e01-bb2e-43f32a15b7ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b12c0672cab44c8ba18ee5e1f60c8659'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:55.897545 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-1966a751-a610-4e01-bb2e-43f32a15b7ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b12c0672cab44c8ba18ee5e1f60c8659'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:55.898195 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-1966a751-a610-4e01-bb2e-43f32a15b7ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b12c0672cab44c8ba18ee5e1f60c8659'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:55.944423 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 946/1882] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 13:46:55 2022] DELETE /identity/v3/users/b12c0672cab44c8ba18ee5e1f60c8659 => generated 0 bytes in 719 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:46:55.944975 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24877dd-e04b-48c8-8e2c-7e73f4bd52ac admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:55.948868 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbedb110-8372-4552-bf3c-0f76efe6d2c6 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:55.949164 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbedb110-8372-4552-bf3c-0f76efe6d2c6 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:55.950325 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbedb110-8372-4552-bf3c-0f76efe6d2c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:56.061600 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb48835-ee2c-4b2f-b341-be7187baef81 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c69161cd520b4b29bf50f3bf2f1c033b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:56.063672 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb48835-ee2c-4b2f-b341-be7187baef81 service nova] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:56.064065 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb48835-ee2c-4b2f-b341-be7187baef81 service nova] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:56.064482 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb48835-ee2c-4b2f-b341-be7187baef81 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:56.067667 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-7bb48835-ee2c-4b2f-b341-be7187baef81 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:56.076617 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-dbedb110-8372-4552-bf3c-0f76efe6d2c6 None None] MFA Rules not processed for user `b54341f7a6ab4450b53924664cb4732a`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:56.078197 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-dbedb110-8372-4552-bf3c-0f76efe6d2c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:56.087005 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb48835-ee2c-4b2f-b341-be7187baef81 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:56.096199 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb48835-ee2c-4b2f-b341-be7187baef81 service nova] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:56.121329 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 937/1883] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:46:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:46:56.252866 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 947/1884] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:46:55 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:46:56.264882 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb48835-ee2c-4b2f-b341-be7187baef81 service nova] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:56.272311 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-7bb48835-ee2c-4b2f-b341-be7187baef81 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:56.365695 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb48835-ee2c-4b2f-b341-be7187baef81 service nova] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:56.369320 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33de591e-6ec1-43b3-9255-487c6bd85f69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:56.369981 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33de591e-6ec1-43b3-9255-487c6bd85f69 service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:56.370717 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33de591e-6ec1-43b3-9255-487c6bd85f69 service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:56.370843 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33de591e-6ec1-43b3-9255-487c6bd85f69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:56.383025 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33de591e-6ec1-43b3-9255-487c6bd85f69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:56.384460 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33de591e-6ec1-43b3-9255-487c6bd85f69 service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:56.395844 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 938/1885] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:46:56 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:46:56.507711 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbedb110-8372-4552-bf3c-0f76efe6d2c6 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:56.523030 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbedb110-8372-4552-bf3c-0f76efe6d2c6 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:56.527290 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9ba1ab-0960-4f66-bd03-f1f667d56dcc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:56.527982 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9ba1ab-0960-4f66-bd03-f1f667d56dcc service cinder] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:56.528374 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9ba1ab-0960-4f66-bd03-f1f667d56dcc service cinder] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:56.529183 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9ba1ab-0960-4f66-bd03-f1f667d56dcc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:56.532262 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-7f9ba1ab-0960-4f66-bd03-f1f667d56dcc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:56.635065 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9ba1ab-0960-4f66-bd03-f1f667d56dcc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:56.637362 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9ba1ab-0960-4f66-bd03-f1f667d56dcc service cinder] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:56.666017 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 948/1886] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:46:56 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:46:56.695732 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc175320-9301-4a8a-8d22-d4867eda5597 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:56.695732 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc175320-9301-4a8a-8d22-d4867eda5597 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:56.696005 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc175320-9301-4a8a-8d22-d4867eda5597 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:56.697773 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 939/1887] 10.222.0.58 () {54 vars in 921 bytes} [Tue Jul 26 13:46:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:46:56.710033 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da72c2a-b801-4708-923f-a31e806bbef3 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:56.711144 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da72c2a-b801-4708-923f-a31e806bbef3 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:56.711686 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da72c2a-b801-4708-923f-a31e806bbef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:56.781222 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-6da72c2a-b801-4708-923f-a31e806bbef3 None None] MFA Rules not processed for user `c69161cd520b4b29bf50f3bf2f1c033b`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:56.788359 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-6da72c2a-b801-4708-923f-a31e806bbef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:56.821024 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 949/1888] 10.222.0.58 () {58 vars in 1018 bytes} [Tue Jul 26 13:46:56 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:46:56.928662 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc175320-9301-4a8a-8d22-d4867eda5597 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:56.930778 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-dc175320-9301-4a8a-8d22-d4867eda5597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:56.959399 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc175320-9301-4a8a-8d22-d4867eda5597 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:56.966117 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a21003-d786-48e4-b845-dc1314dca500 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c69161cd520b4b29bf50f3bf2f1c033b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:56.966906 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a21003-d786-48e4-b845-dc1314dca500 service nova] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:56.967293 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a21003-d786-48e4-b845-dc1314dca500 service nova] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:56.967778 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a21003-d786-48e4-b845-dc1314dca500 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:56.979165 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a21003-d786-48e4-b845-dc1314dca500 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:56.981542 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a21003-d786-48e4-b845-dc1314dca500 service nova] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:57.003745 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 940/1889] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:46:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:46:58.210147 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da72c2a-b801-4708-923f-a31e806bbef3 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:58.211704 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-6da72c2a-b801-4708-923f-a31e806bbef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:58.266870 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:58.283148 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da72c2a-b801-4708-923f-a31e806bbef3 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:58.290266 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380f0b3d-4817-4df4-9bd5-fe23bbdfc64f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:58.291252 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380f0b3d-4817-4df4-9bd5-fe23bbdfc64f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:58.291940 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380f0b3d-4817-4df4-9bd5-fe23bbdfc64f admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:58.292269 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380f0b3d-4817-4df4-9bd5-fe23bbdfc64f admin admin] PATH_INFO: `/v3/users/e1852137064c41a8af6ba370eea4e35b` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:58.293538 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380f0b3d-4817-4df4-9bd5-fe23bbdfc64f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1852137064c41a8af6ba370eea4e35b)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:58.295550 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380f0b3d-4817-4df4-9bd5-fe23bbdfc64f admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:58.347556 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a21003-d786-48e4-b845-dc1314dca500 service nova] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:58.349690 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-f9a21003-d786-48e4-b845-dc1314dca500 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:58.354277 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-380f0b3d-4817-4df4-9bd5-fe23bbdfc64f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1852137064c41a8af6ba370eea4e35b'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:58.363454 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-380f0b3d-4817-4df4-9bd5-fe23bbdfc64f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1852137064c41a8af6ba370eea4e35b'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:58.364679 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a21003-d786-48e4-b845-dc1314dca500 service nova] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:58.369442 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6180950b-40a7-42af-a217-b7b3fbd2b08b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:58.370131 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6180950b-40a7-42af-a217-b7b3fbd2b08b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:58.370414 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6180950b-40a7-42af-a217-b7b3fbd2b08b admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:58.370846 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6180950b-40a7-42af-a217-b7b3fbd2b08b admin admin] PATH_INFO: `/v3/projects/18fa4c5418c8450aa27cdf1069ab8d22` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:58.372566 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6180950b-40a7-42af-a217-b7b3fbd2b08b admin admin] RBAC: Authorizing `identity:delete_project(project_id=18fa4c5418c8450aa27cdf1069ab8d22)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:58.372699 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-380f0b3d-4817-4df4-9bd5-fe23bbdfc64f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1852137064c41a8af6ba370eea4e35b'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:58.373449 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-380f0b3d-4817-4df4-9bd5-fe23bbdfc64f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1852137064c41a8af6ba370eea4e35b'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:58.374676 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6180950b-40a7-42af-a217-b7b3fbd2b08b admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:58.375594 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-380f0b3d-4817-4df4-9bd5-fe23bbdfc64f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1852137064c41a8af6ba370eea4e35b'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:58.382918 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:58.401595 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 950/1890] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 13:46:58 2022] DELETE /identity/v3/users/e1852137064c41a8af6ba370eea4e35b => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:46:58.404629 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63248d41-c83a-4918-a0fc-47a8ec3fdc88 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:58.405805 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63248d41-c83a-4918-a0fc-47a8ec3fdc88 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:58.406356 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63248d41-c83a-4918-a0fc-47a8ec3fdc88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:58.428604 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-63248d41-c83a-4918-a0fc-47a8ec3fdc88 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:58.439224 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-6180950b-40a7-42af-a217-b7b3fbd2b08b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18fa4c5418c8450aa27cdf1069ab8d22', 'request_id': 'req-6180950b-40a7-42af-a217-b7b3fbd2b08b'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:58.442042 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-63248d41-c83a-4918-a0fc-47a8ec3fdc88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:58.449986 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-6180950b-40a7-42af-a217-b7b3fbd2b08b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18fa4c5418c8450aa27cdf1069ab8d22', 'request_id': 'req-6180950b-40a7-42af-a217-b7b3fbd2b08b'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:58.458246 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-6180950b-40a7-42af-a217-b7b3fbd2b08b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18fa4c5418c8450aa27cdf1069ab8d22', 'request_id': 'req-6180950b-40a7-42af-a217-b7b3fbd2b08b'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:58.476454 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-6180950b-40a7-42af-a217-b7b3fbd2b08b admin admin] The token cache is being invalidate because project 18fa4c5418c8450aa27cdf1069ab8d22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 13:46:58.477080 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-6180950b-40a7-42af-a217-b7b3fbd2b08b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:58.478290 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-6180950b-40a7-42af-a217-b7b3fbd2b08b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:58.484036 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 941/1891] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 13:46:58 2022] DELETE /identity/v3/projects/18fa4c5418c8450aa27cdf1069ab8d22 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 13:46:58.486308 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff9626a-65d6-46f5-8770-ae6fd4b9a164 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:58.486603 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff9626a-65d6-46f5-8770-ae6fd4b9a164 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:58.486875 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff9626a-65d6-46f5-8770-ae6fd4b9a164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:58.534195 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-3ff9626a-65d6-46f5-8770-ae6fd4b9a164 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:58.537430 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-3ff9626a-65d6-46f5-8770-ae6fd4b9a164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:58.543551 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 951/1892] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:46:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 13:46:58.551270 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63248d41-c83a-4918-a0fc-47a8ec3fdc88 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:58.552859 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-63248d41-c83a-4918-a0fc-47a8ec3fdc88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:58.561257 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 942/1893] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:46:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:46:58.563430 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996ccedc-bbce-4d29-813b-d090da6ccdd8 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:58.563727 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996ccedc-bbce-4d29-813b-d090da6ccdd8 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:58.564011 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996ccedc-bbce-4d29-813b-d090da6ccdd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:58.612516 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-996ccedc-bbce-4d29-813b-d090da6ccdd8 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:58.615781 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-996ccedc-bbce-4d29-813b-d090da6ccdd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:58.626763 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63248d41-c83a-4918-a0fc-47a8ec3fdc88 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:58.630780 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc8fd07-fa77-4229-90f5-60ad7f278719 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:58.631504 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc8fd07-fa77-4229-90f5-60ad7f278719 service neutron] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:58.631854 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc8fd07-fa77-4229-90f5-60ad7f278719 service neutron] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:58.632224 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc8fd07-fa77-4229-90f5-60ad7f278719 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:58.643747 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc8fd07-fa77-4229-90f5-60ad7f278719 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:58.644889 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc8fd07-fa77-4229-90f5-60ad7f278719 service neutron] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:58.650488 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 943/1894] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:46:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:46:58.651599 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996ccedc-bbce-4d29-813b-d090da6ccdd8 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:58.669618 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 952/1895] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:46:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 13:46:58.676813 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996ccedc-bbce-4d29-813b-d090da6ccdd8 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:58.688656 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d021f5d-47c1-4af9-88f1-03a81a9ef41c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:58.689652 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d021f5d-47c1-4af9-88f1-03a81a9ef41c admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:58.689652 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d021f5d-47c1-4af9-88f1-03a81a9ef41c admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:58.689870 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d021f5d-47c1-4af9-88f1-03a81a9ef41c admin admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:58.690711 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d021f5d-47c1-4af9-88f1-03a81a9ef41c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:46:58.691034 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d021f5d-47c1-4af9-88f1-03a81a9ef41c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:58.693798 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d021f5d-47c1-4af9-88f1-03a81a9ef41c admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:58.719567 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 944/1896] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Jul 26 13:46:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:46:58.719683 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc8fd07-fa77-4229-90f5-60ad7f278719 service neutron] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:58.732290 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc8fd07-fa77-4229-90f5-60ad7f278719 service neutron] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:58.744822 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c780f424-9f41-4f8a-81a8-13435585a783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:58.745577 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c780f424-9f41-4f8a-81a8-13435585a783 admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:58.745964 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c780f424-9f41-4f8a-81a8-13435585a783 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:58.748750 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c780f424-9f41-4f8a-81a8-13435585a783 admin admin] PATH_INFO: `/v3/projects` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:58.749511 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c780f424-9f41-4f8a-81a8-13435585a783 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:58.752125 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c780f424-9f41-4f8a-81a8-13435585a783 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:58.783000 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:58.832458 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 953/1897] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 13:46:58 2022] POST /identity/v3/projects => generated 387 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:46:58.838628 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d021f5d-47c1-4af9-88f1-03a81a9ef41c admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:58.871363 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d021f5d-47c1-4af9-88f1-03a81a9ef41c admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:58.998263 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7260509b-ca60-4201-83e3-943a88a0c5fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:58.999018 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7260509b-ca60-4201-83e3-943a88a0c5fd admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:58.999310 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7260509b-ca60-4201-83e3-943a88a0c5fd admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:58.999614 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7260509b-ca60-4201-83e3-943a88a0c5fd admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:59.000545 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7260509b-ca60-4201-83e3-943a88a0c5fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:59.004745 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7260509b-ca60-4201-83e3-943a88a0c5fd admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:59.095150 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 945/1898] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 13:46:58 2022] POST /identity/v3/users => generated 294 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:46:59.100113 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c780f424-9f41-4f8a-81a8-13435585a783 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:59.119186 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c780f424-9f41-4f8a-81a8-13435585a783 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:59.137876 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf7bb76-edcc-4c4c-99a1-49c02788aa63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:59.138549 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf7bb76-edcc-4c4c-99a1-49c02788aa63 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:59.138838 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf7bb76-edcc-4c4c-99a1-49c02788aa63 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:59.139121 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf7bb76-edcc-4c4c-99a1-49c02788aa63 admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:59.139833 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf7bb76-edcc-4c4c-99a1-49c02788aa63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:59.142107 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf7bb76-edcc-4c4c-99a1-49c02788aa63 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:59.166410 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 954/1899] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:46:59 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:46:59.170904 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7260509b-ca60-4201-83e3-943a88a0c5fd admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:59.195247 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7260509b-ca60-4201-83e3-943a88a0c5fd admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:59.201417 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8163e130-94be-4f69-ad72-cbbf7e2d61c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:59.201762 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8163e130-94be-4f69-ad72-cbbf7e2d61c0 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:59.202151 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8163e130-94be-4f69-ad72-cbbf7e2d61c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:59.202428 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8163e130-94be-4f69-ad72-cbbf7e2d61c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:59.204088 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8163e130-94be-4f69-ad72-cbbf7e2d61c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:59.205767 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8163e130-94be-4f69-ad72-cbbf7e2d61c0 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:59.224028 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 946/1900] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:46:59 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:46:59.228540 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf7bb76-edcc-4c4c-99a1-49c02788aa63 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:59.278745 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf7bb76-edcc-4c4c-99a1-49c02788aa63 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:59.296411 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5aa88b-6f97-439a-957a-fc9f0af97c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:59.299939 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5aa88b-6f97-439a-957a-fc9f0af97c19 admin admin] REQUEST_METHOD: `PUT` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:59.300238 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5aa88b-6f97-439a-957a-fc9f0af97c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:59.300517 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5aa88b-6f97-439a-957a-fc9f0af97c19 admin admin] PATH_INFO: `/v3/projects/12536ebe236543c8825d177f03048642/users/11477143e1184c8cb6a584550690ed43/roles/759f389b221049c4b01faf3b984cd8d7` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:59.338970 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5aa88b-6f97-439a-957a-fc9f0af97c19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12536ebe236543c8825d177f03048642, user_id=11477143e1184c8cb6a584550690ed43, role_id=759f389b221049c4b01faf3b984cd8d7)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:59.343856 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5aa88b-6f97-439a-957a-fc9f0af97c19 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:59.369877 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 955/1901] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 13:46:59 2022] PUT /identity/v3/projects/12536ebe236543c8825d177f03048642/users/11477143e1184c8cb6a584550690ed43/roles/759f389b221049c4b01faf3b984cd8d7 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:46:59.381191 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3329321d-fd6c-4670-aa67-39af9222d7f6 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:59.381666 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3329321d-fd6c-4670-aa67-39af9222d7f6 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:59.381758 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3329321d-fd6c-4670-aa67-39af9222d7f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:59.428159 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-3329321d-fd6c-4670-aa67-39af9222d7f6 None None] MFA Rules not processed for user `11477143e1184c8cb6a584550690ed43`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:46:59.429394 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-3329321d-fd6c-4670-aa67-39af9222d7f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:59.489305 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:59.544237 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 947/1902] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:46:59 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:46:59.549427 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5aa88b-6f97-439a-957a-fc9f0af97c19 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:59.597050 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3329321d-fd6c-4670-aa67-39af9222d7f6 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:59.598833 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-3329321d-fd6c-4670-aa67-39af9222d7f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:59.650911 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5aa88b-6f97-439a-957a-fc9f0af97c19 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:59.659145 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a820ce-ec3b-4c3c-a189-4f4047507da5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:59.659832 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a820ce-ec3b-4c3c-a189-4f4047507da5 admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:59.660120 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a820ce-ec3b-4c3c-a189-4f4047507da5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:59.660423 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a820ce-ec3b-4c3c-a189-4f4047507da5 admin admin] PATH_INFO: `/v3/projects` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:59.661255 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a820ce-ec3b-4c3c-a189-4f4047507da5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:59.663224 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a820ce-ec3b-4c3c-a189-4f4047507da5 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:59.712022 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 956/1903] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 13:46:59 2022] POST /identity/v3/projects => generated 385 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:46:59.717494 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a820ce-ec3b-4c3c-a189-4f4047507da5 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:59.720020 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3329321d-fd6c-4670-aa67-39af9222d7f6 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:59.807770 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a820ce-ec3b-4c3c-a189-4f4047507da5 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:59.807942 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7049609-aa9a-4661-b83e-5b0385286f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:59.808725 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7049609-aa9a-4661-b83e-5b0385286f52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:59.809159 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7049609-aa9a-4661-b83e-5b0385286f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:59.809576 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7049609-aa9a-4661-b83e-5b0385286f52 admin admin] PATH_INFO: `/v3/projects/78ec27f47dec4361bec73ff96578ab39` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:59.810979 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7049609-aa9a-4661-b83e-5b0385286f52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78ec27f47dec4361bec73ff96578ab39)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:59.813257 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc03a8d1-b387-4f71-ad61-a95272e07e05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:59.813408 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7049609-aa9a-4661-b83e-5b0385286f52 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:59.814057 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc03a8d1-b387-4f71-ad61-a95272e07e05 admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:46:59.814411 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc03a8d1-b387-4f71-ad61-a95272e07e05 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:46:59.814756 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc03a8d1-b387-4f71-ad61-a95272e07e05 admin admin] PATH_INFO: `/v3/users` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:46:59.815816 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc03a8d1-b387-4f71-ad61-a95272e07e05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:46:59.818309 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc03a8d1-b387-4f71-ad61-a95272e07e05 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:46:59.857524 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 957/1904] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 13:46:59 2022] POST /identity/v3/users => generated 299 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 13:46:59.862297 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc03a8d1-b387-4f71-ad61-a95272e07e05 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:59.884848 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc03a8d1-b387-4f71-ad61-a95272e07e05 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:46:59.912867 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-b7049609-aa9a-4661-b83e-5b0385286f52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78ec27f47dec4361bec73ff96578ab39', 'request_id': 'req-b7049609-aa9a-4661-b83e-5b0385286f52'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:59.925512 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-b7049609-aa9a-4661-b83e-5b0385286f52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78ec27f47dec4361bec73ff96578ab39', 'request_id': 'req-b7049609-aa9a-4661-b83e-5b0385286f52'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:59.929405 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:59.935973 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-b7049609-aa9a-4661-b83e-5b0385286f52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78ec27f47dec4361bec73ff96578ab39', 'request_id': 'req-b7049609-aa9a-4661-b83e-5b0385286f52'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:59.960512 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-b7049609-aa9a-4661-b83e-5b0385286f52 admin admin] The token cache is being invalidate because project 78ec27f47dec4361bec73ff96578ab39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 13:46:59.961259 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-b7049609-aa9a-4661-b83e-5b0385286f52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:59.962520 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-b7049609-aa9a-4661-b83e-5b0385286f52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:46:59.968252 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 948/1905] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 13:46:59 2022] DELETE /identity/v3/projects/78ec27f47dec4361bec73ff96578ab39 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:46:59.970340 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7049609-aa9a-4661-b83e-5b0385286f52 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:46:59.972215 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-b7049609-aa9a-4661-b83e-5b0385286f52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:46:59.995639 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:46:59.997787 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab1adbb-e4f3-40cd-8ee8-5068289672d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:46:59.999909 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab1adbb-e4f3-40cd-8ee8-5068289672d2 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:00.001896 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab1adbb-e4f3-40cd-8ee8-5068289672d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:00.002181 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab1adbb-e4f3-40cd-8ee8-5068289672d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:00.003086 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab1adbb-e4f3-40cd-8ee8-5068289672d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:00.009168 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab1adbb-e4f3-40cd-8ee8-5068289672d2 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:00.041145 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 958/1906] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:46:59 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 13:47:00.044706 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab1adbb-e4f3-40cd-8ee8-5068289672d2 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:00.048673 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-eab1adbb-e4f3-40cd-8ee8-5068289672d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:00.083387 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab1adbb-e4f3-40cd-8ee8-5068289672d2 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:00.113964 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc7d74b-8e05-4cc2-9371-b82f31771b4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:00.116109 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc7d74b-8e05-4cc2-9371-b82f31771b4d admin admin] REQUEST_METHOD: `PUT` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:00.116394 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc7d74b-8e05-4cc2-9371-b82f31771b4d admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:00.116675 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc7d74b-8e05-4cc2-9371-b82f31771b4d admin admin] PATH_INFO: `/v3/projects/0431e54a61d44cf3bfeaa6681788c337/users/1e0e0faff01640ac9bf4763b615b5204/roles/6e0201a9a6484f7d8b112b1eafff70bd` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:00.146979 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7049609-aa9a-4661-b83e-5b0385286f52 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:00.159965 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc7d74b-8e05-4cc2-9371-b82f31771b4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0431e54a61d44cf3bfeaa6681788c337, user_id=1e0e0faff01640ac9bf4763b615b5204, role_id=6e0201a9a6484f7d8b112b1eafff70bd)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:00.162930 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc7d74b-8e05-4cc2-9371-b82f31771b4d admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:00.174746 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98db7112-ab5c-4ef0-8fec-ca76c3ffcf19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:00.177160 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98db7112-ab5c-4ef0-8fec-ca76c3ffcf19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:00.177995 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98db7112-ab5c-4ef0-8fec-ca76c3ffcf19 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:00.178605 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98db7112-ab5c-4ef0-8fec-ca76c3ffcf19 admin admin] PATH_INFO: `/v3/users/5bda2e864e524ad28ae9f165058c463d` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:00.180210 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98db7112-ab5c-4ef0-8fec-ca76c3ffcf19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5bda2e864e524ad28ae9f165058c463d)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:00.182794 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 959/1907] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 13:47:00 2022] PUT /identity/v3/projects/0431e54a61d44cf3bfeaa6681788c337/users/1e0e0faff01640ac9bf4763b615b5204/roles/6e0201a9a6484f7d8b112b1eafff70bd => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 13:47:00.183002 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98db7112-ab5c-4ef0-8fec-ca76c3ffcf19 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:00.186890 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc7d74b-8e05-4cc2-9371-b82f31771b4d admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:00.353799 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-98db7112-ab5c-4ef0-8fec-ca76c3ffcf19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5bda2e864e524ad28ae9f165058c463d'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:00.353799 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-98db7112-ab5c-4ef0-8fec-ca76c3ffcf19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5bda2e864e524ad28ae9f165058c463d'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:00.355547 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc7d74b-8e05-4cc2-9371-b82f31771b4d admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:00.371961 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-98db7112-ab5c-4ef0-8fec-ca76c3ffcf19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5bda2e864e524ad28ae9f165058c463d'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:00.372779 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-98db7112-ab5c-4ef0-8fec-ca76c3ffcf19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5bda2e864e524ad28ae9f165058c463d'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:00.382522 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390f0a7e-96d7-43b9-ba1e-ff594da9665c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:00.383171 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390f0a7e-96d7-43b9-ba1e-ff594da9665c admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:00.383456 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390f0a7e-96d7-43b9-ba1e-ff594da9665c admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:00.384305 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390f0a7e-96d7-43b9-ba1e-ff594da9665c admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:00.385195 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390f0a7e-96d7-43b9-ba1e-ff594da9665c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:00.387586 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390f0a7e-96d7-43b9-ba1e-ff594da9665c admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:00.401097 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 960/1908] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:47:00 2022] GET /identity/v3/roles => generated 1420 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 13:47:00.405671 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390f0a7e-96d7-43b9-ba1e-ff594da9665c admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:00.407628 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-390f0a7e-96d7-43b9-ba1e-ff594da9665c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:00.408482 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-98db7112-ab5c-4ef0-8fec-ca76c3ffcf19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5bda2e864e524ad28ae9f165058c463d'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:00.423617 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390f0a7e-96d7-43b9-ba1e-ff594da9665c admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:00.431590 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637b8315-2ac2-4865-8401-08408e9abd8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:00.432451 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637b8315-2ac2-4865-8401-08408e9abd8a admin admin] REQUEST_METHOD: `PUT` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:00.432885 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637b8315-2ac2-4865-8401-08408e9abd8a admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:00.433292 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637b8315-2ac2-4865-8401-08408e9abd8a admin admin] PATH_INFO: `/v3/projects/0431e54a61d44cf3bfeaa6681788c337/users/1e0e0faff01640ac9bf4763b615b5204/roles/6e0201a9a6484f7d8b112b1eafff70bd` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:00.436925 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637b8315-2ac2-4865-8401-08408e9abd8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0431e54a61d44cf3bfeaa6681788c337, user_id=1e0e0faff01640ac9bf4763b615b5204, role_id=6e0201a9a6484f7d8b112b1eafff70bd)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:00.440987 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637b8315-2ac2-4865-8401-08408e9abd8a admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:00.455164 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 949/1909] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 13:46:59 2022] DELETE /identity/v3/users/5bda2e864e524ad28ae9f165058c463d => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:47:00.493377 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 961/1910] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 13:47:00 2022] PUT /identity/v3/projects/0431e54a61d44cf3bfeaa6681788c337/users/1e0e0faff01640ac9bf4763b615b5204/roles/6e0201a9a6484f7d8b112b1eafff70bd => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:47:00.513378 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744cbcb5-5a4e-4c21-919d-0fd17b4b2334 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:00.513378 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744cbcb5-5a4e-4c21-919d-0fd17b4b2334 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:00.531552 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744cbcb5-5a4e-4c21-919d-0fd17b4b2334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:00.628250 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-744cbcb5-5a4e-4c21-919d-0fd17b4b2334 None None] MFA Rules not processed for user `1e0e0faff01640ac9bf4763b615b5204`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:47:00.629572 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-744cbcb5-5a4e-4c21-919d-0fd17b4b2334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:00.842208 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 950/1911] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:47:00 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:47:00.853810 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637b8315-2ac2-4865-8401-08408e9abd8a admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:00.858929 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-637b8315-2ac2-4865-8401-08408e9abd8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:00.962758 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637b8315-2ac2-4865-8401-08408e9abd8a admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:01.022759 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:47:01.037547 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99055211-d4e1-4bc4-870c-f4ab1a1c65a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:01.038830 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99055211-d4e1-4bc4-870c-f4ab1a1c65a9 service cinder] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:01.039220 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99055211-d4e1-4bc4-870c-f4ab1a1c65a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:01.039751 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99055211-d4e1-4bc4-870c-f4ab1a1c65a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:01.052268 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99055211-d4e1-4bc4-870c-f4ab1a1c65a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:01.054353 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99055211-d4e1-4bc4-870c-f4ab1a1c65a9 service cinder] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:01.069981 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 962/1912] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:47:00 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:47:02.045775 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744cbcb5-5a4e-4c21-919d-0fd17b4b2334 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:02.083973 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:47:02.151941 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744cbcb5-5a4e-4c21-919d-0fd17b4b2334 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:02.168016 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee920899-9a7a-4165-aa15-bf6cb505a575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:02.169780 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee920899-9a7a-4165-aa15-bf6cb505a575 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:02.170066 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee920899-9a7a-4165-aa15-bf6cb505a575 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:02.170530 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee920899-9a7a-4165-aa15-bf6cb505a575 admin admin] PATH_INFO: `/v3/users/11477143e1184c8cb6a584550690ed43` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:02.172732 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee920899-9a7a-4165-aa15-bf6cb505a575 admin admin] RBAC: Authorizing `identity:delete_user(user_id=11477143e1184c8cb6a584550690ed43)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:02.175239 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee920899-9a7a-4165-aa15-bf6cb505a575 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:02.198896 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99055211-d4e1-4bc4-870c-f4ab1a1c65a9 service cinder] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:02.201928 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-99055211-d4e1-4bc4-870c-f4ab1a1c65a9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:02.227624 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99055211-d4e1-4bc4-870c-f4ab1a1c65a9 service cinder] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:02.251620 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577e2210-c0e8-4df1-9496-36d765b8ec1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:02.252992 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577e2210-c0e8-4df1-9496-36d765b8ec1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:02.257044 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577e2210-c0e8-4df1-9496-36d765b8ec1e admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:02.257510 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577e2210-c0e8-4df1-9496-36d765b8ec1e admin admin] PATH_INFO: `/v3/projects/4aa9fe91588547b5887bc5333a782f81` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:02.262275 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577e2210-c0e8-4df1-9496-36d765b8ec1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=4aa9fe91588547b5887bc5333a782f81)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:02.269181 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577e2210-c0e8-4df1-9496-36d765b8ec1e admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:02.287564 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-ee920899-9a7a-4165-aa15-bf6cb505a575 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11477143e1184c8cb6a584550690ed43'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:02.288364 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-ee920899-9a7a-4165-aa15-bf6cb505a575 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11477143e1184c8cb6a584550690ed43'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:02.302989 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-ee920899-9a7a-4165-aa15-bf6cb505a575 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11477143e1184c8cb6a584550690ed43'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:02.304013 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-ee920899-9a7a-4165-aa15-bf6cb505a575 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11477143e1184c8cb6a584550690ed43'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:02.339869 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-ee920899-9a7a-4165-aa15-bf6cb505a575 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11477143e1184c8cb6a584550690ed43'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:02.365807 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 951/1913] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 13:47:02 2022] DELETE /identity/v3/users/11477143e1184c8cb6a584550690ed43 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:47:02.370971 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31547c9f-f42f-4530-9bb6-571b26f14003 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:02.372037 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31547c9f-f42f-4530-9bb6-571b26f14003 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:02.372529 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31547c9f-f42f-4530-9bb6-571b26f14003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:02.377573 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-577e2210-c0e8-4df1-9496-36d765b8ec1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4aa9fe91588547b5887bc5333a782f81', 'request_id': 'req-577e2210-c0e8-4df1-9496-36d765b8ec1e'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:02.394257 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-577e2210-c0e8-4df1-9496-36d765b8ec1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4aa9fe91588547b5887bc5333a782f81', 'request_id': 'req-577e2210-c0e8-4df1-9496-36d765b8ec1e'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:02.401516 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-577e2210-c0e8-4df1-9496-36d765b8ec1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4aa9fe91588547b5887bc5333a782f81', 'request_id': 'req-577e2210-c0e8-4df1-9496-36d765b8ec1e'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:02.411495 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-31547c9f-f42f-4530-9bb6-571b26f14003 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:47:02.413385 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-31547c9f-f42f-4530-9bb6-571b26f14003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:02.424099 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-577e2210-c0e8-4df1-9496-36d765b8ec1e admin admin] The token cache is being invalidate because project 4aa9fe91588547b5887bc5333a782f81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86933) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 13:47:02.424336 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-577e2210-c0e8-4df1-9496-36d765b8ec1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:02.433011 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-577e2210-c0e8-4df1-9496-36d765b8ec1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:02.434764 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 963/1914] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 13:47:02 2022] DELETE /identity/v3/projects/4aa9fe91588547b5887bc5333a782f81 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:47:02.440101 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7a2aec-e74a-46d5-a86e-91eb69d33fe3 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:02.440615 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7a2aec-e74a-46d5-a86e-91eb69d33fe3 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:02.441055 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7a2aec-e74a-46d5-a86e-91eb69d33fe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:02.462191 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-6f7a2aec-e74a-46d5-a86e-91eb69d33fe3 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:47:02.463991 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-6f7a2aec-e74a-46d5-a86e-91eb69d33fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:02.535502 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 952/1915] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:47:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:47:02.592997 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31547c9f-f42f-4530-9bb6-571b26f14003 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:02.595759 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-31547c9f-f42f-4530-9bb6-571b26f14003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:02.596764 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 964/1916] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:47:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:47:02.604713 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e03ca9d-e996-4172-a806-481d8dfe087e None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:02.605057 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e03ca9d-e996-4172-a806-481d8dfe087e None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:02.605349 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e03ca9d-e996-4172-a806-481d8dfe087e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:02.657815 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-6e03ca9d-e996-4172-a806-481d8dfe087e None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:47:02.663777 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-6e03ca9d-e996-4172-a806-481d8dfe087e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:02.693849 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:47:02.701547 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 965/1917] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:47:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 13:47:02.702404 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ada2e1-63a6-4ce2-8ede-c360982cde86 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:02.703710 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ada2e1-63a6-4ce2-8ede-c360982cde86 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:02.705711 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ada2e1-63a6-4ce2-8ede-c360982cde86 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:02.707786 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 966/1918] 10.222.0.58 () {54 vars in 921 bytes} [Tue Jul 26 13:47:02 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:47:02.711727 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ada2e1-63a6-4ce2-8ede-c360982cde86 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:02.726527 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31547c9f-f42f-4530-9bb6-571b26f14003 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:02.732315 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff428fc1-a37c-4616-a86f-4d0b631db87d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:02.734017 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff428fc1-a37c-4616-a86f-4d0b631db87d service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:02.734669 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff428fc1-a37c-4616-a86f-4d0b631db87d service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:02.735804 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff428fc1-a37c-4616-a86f-4d0b631db87d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:02.739978 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ada2e1-63a6-4ce2-8ede-c360982cde86 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:02.748207 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf155d5a-9add-47be-ac6d-b6c17b9ef902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:02.749386 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf155d5a-9add-47be-ac6d-b6c17b9ef902 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:02.750102 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff428fc1-a37c-4616-a86f-4d0b631db87d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:02.750469 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf155d5a-9add-47be-ac6d-b6c17b9ef902 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:02.750802 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf155d5a-9add-47be-ac6d-b6c17b9ef902 admin admin] PATH_INFO: `/v3/domains` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:02.751448 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff428fc1-a37c-4616-a86f-4d0b631db87d service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:02.751783 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf155d5a-9add-47be-ac6d-b6c17b9ef902 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:47:02.752856 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf155d5a-9add-47be-ac6d-b6c17b9ef902 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:02.754760 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf155d5a-9add-47be-ac6d-b6c17b9ef902 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:02.776759 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 953/1919] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:47:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 13:47:02.777703 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 967/1920] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Jul 26 13:47:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:47:02.779302 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf155d5a-9add-47be-ac6d-b6c17b9ef902 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:02.781318 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d02f341-fd3f-4ceb-8429-58a0b4e2a210 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:02.781747 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d02f341-fd3f-4ceb-8429-58a0b4e2a210 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:02.782032 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d02f341-fd3f-4ceb-8429-58a0b4e2a210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:02.815335 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf155d5a-9add-47be-ac6d-b6c17b9ef902 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:02.826580 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-9d02f341-fd3f-4ceb-8429-58a0b4e2a210 None None] MFA Rules not processed for user `c69161cd520b4b29bf50f3bf2f1c033b`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:47:02.829080 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65861a8c-b673-4fe0-ae2c-37f922bee9ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:02.830190 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65861a8c-b673-4fe0-ae2c-37f922bee9ce admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:02.830829 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65861a8c-b673-4fe0-ae2c-37f922bee9ce admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:02.831300 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65861a8c-b673-4fe0-ae2c-37f922bee9ce admin admin] PATH_INFO: `/v3/projects` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:02.832215 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-9d02f341-fd3f-4ceb-8429-58a0b4e2a210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:02.833048 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65861a8c-b673-4fe0-ae2c-37f922bee9ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:02.837341 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65861a8c-b673-4fe0-ae2c-37f922bee9ce admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:02.906281 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 968/1921] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 13:47:02 2022] POST /identity/v3/projects => generated 387 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:47:02.910437 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65861a8c-b673-4fe0-ae2c-37f922bee9ce admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:02.991716 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 954/1922] 10.222.0.58 () {58 vars in 1018 bytes} [Tue Jul 26 13:47:02 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:47:02.997897 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:47:03.012976 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d02f341-fd3f-4ceb-8429-58a0b4e2a210 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:03.014932 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-9d02f341-fd3f-4ceb-8429-58a0b4e2a210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:03.051461 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d02f341-fd3f-4ceb-8429-58a0b4e2a210 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:03.069875 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65861a8c-b673-4fe0-ae2c-37f922bee9ce admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:03.084795 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5534140-631d-44c9-9786-cf8826dc8a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:03.085457 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5534140-631d-44c9-9786-cf8826dc8a1e admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:03.085786 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5534140-631d-44c9-9786-cf8826dc8a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:03.086059 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5534140-631d-44c9-9786-cf8826dc8a1e admin admin] PATH_INFO: `/v3/users` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:03.086933 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5534140-631d-44c9-9786-cf8826dc8a1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:03.089511 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5534140-631d-44c9-9786-cf8826dc8a1e admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:03.126339 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 969/1923] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 13:47:02 2022] POST /identity/v3/users => generated 294 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 13:47:03.127895 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5534140-631d-44c9-9786-cf8826dc8a1e admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:03.170268 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5534140-631d-44c9-9786-cf8826dc8a1e admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:03.176182 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b070fe76-edaf-4ba0-8446-a70763a00f36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:03.176822 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b070fe76-edaf-4ba0-8446-a70763a00f36 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:03.177107 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b070fe76-edaf-4ba0-8446-a70763a00f36 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:03.177384 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b070fe76-edaf-4ba0-8446-a70763a00f36 admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:03.178328 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b070fe76-edaf-4ba0-8446-a70763a00f36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:03.179934 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b070fe76-edaf-4ba0-8446-a70763a00f36 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:03.191889 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 970/1924] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:47:03 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:47:03.193104 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b070fe76-edaf-4ba0-8446-a70763a00f36 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:03.195438 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-b070fe76-edaf-4ba0-8446-a70763a00f36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:03.200943 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545a5021-9d5f-4307-b3f6-2f3fe5acf312 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c69161cd520b4b29bf50f3bf2f1c033b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:03.201614 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545a5021-9d5f-4307-b3f6-2f3fe5acf312 service nova] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:03.201895 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545a5021-9d5f-4307-b3f6-2f3fe5acf312 service nova] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:03.202176 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545a5021-9d5f-4307-b3f6-2f3fe5acf312 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:03.208521 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b070fe76-edaf-4ba0-8446-a70763a00f36 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:03.212076 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545a5021-9d5f-4307-b3f6-2f3fe5acf312 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:03.213510 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2be0ea-f818-4eee-a716-13d93fc50b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:03.214094 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545a5021-9d5f-4307-b3f6-2f3fe5acf312 service nova] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:03.214301 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2be0ea-f818-4eee-a716-13d93fc50b06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:03.214651 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2be0ea-f818-4eee-a716-13d93fc50b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:03.214992 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2be0ea-f818-4eee-a716-13d93fc50b06 admin admin] PATH_INFO: `/v3/users/aed84db1720b4bb2b1d083f9a8f998f0` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:03.216883 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2be0ea-f818-4eee-a716-13d93fc50b06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aed84db1720b4bb2b1d083f9a8f998f0)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:03.219139 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2be0ea-f818-4eee-a716-13d93fc50b06 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:03.225112 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 955/1925] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:47:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:47:03.225808 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545a5021-9d5f-4307-b3f6-2f3fe5acf312 service nova] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:03.237749 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545a5021-9d5f-4307-b3f6-2f3fe5acf312 service nova] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:03.242655 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ea9b26-875f-4cbb-b37f-536b006ae63e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:03.243416 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ea9b26-875f-4cbb-b37f-536b006ae63e admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:03.243767 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ea9b26-875f-4cbb-b37f-536b006ae63e admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:03.244107 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ea9b26-875f-4cbb-b37f-536b006ae63e admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:03.244865 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ea9b26-875f-4cbb-b37f-536b006ae63e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:03.246751 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ea9b26-875f-4cbb-b37f-536b006ae63e admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:03.260272 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 956/1926] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:47:03 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:47:03.263331 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ea9b26-875f-4cbb-b37f-536b006ae63e admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:03.277191 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ea9b26-875f-4cbb-b37f-536b006ae63e admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:03.284437 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb45a5f-fa1e-4f56-b4df-23773952a038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:03.285215 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb45a5f-fa1e-4f56-b4df-23773952a038 admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:03.285642 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb45a5f-fa1e-4f56-b4df-23773952a038 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:03.286460 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb45a5f-fa1e-4f56-b4df-23773952a038 admin admin] PATH_INFO: `/v3/projects/15cb70e82f804eed9d479538f76ec027/users/f0284f920ee849e4a95b32626395096c/roles/759f389b221049c4b01faf3b984cd8d7` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:03.291887 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-2c2be0ea-f818-4eee-a716-13d93fc50b06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aed84db1720b4bb2b1d083f9a8f998f0'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:03.302875 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-2c2be0ea-f818-4eee-a716-13d93fc50b06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aed84db1720b4bb2b1d083f9a8f998f0'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:03.310522 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb45a5f-fa1e-4f56-b4df-23773952a038 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15cb70e82f804eed9d479538f76ec027, user_id=f0284f920ee849e4a95b32626395096c, role_id=759f389b221049c4b01faf3b984cd8d7)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:03.311514 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-2c2be0ea-f818-4eee-a716-13d93fc50b06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aed84db1720b4bb2b1d083f9a8f998f0'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:03.312268 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-2c2be0ea-f818-4eee-a716-13d93fc50b06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aed84db1720b4bb2b1d083f9a8f998f0'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:03.312976 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-2c2be0ea-f818-4eee-a716-13d93fc50b06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aed84db1720b4bb2b1d083f9a8f998f0'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:03.313427 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb45a5f-fa1e-4f56-b4df-23773952a038 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:03.329301 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 957/1927] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 13:47:03 2022] PUT /identity/v3/projects/15cb70e82f804eed9d479538f76ec027/users/f0284f920ee849e4a95b32626395096c/roles/759f389b221049c4b01faf3b984cd8d7 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:47:03.335991 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 971/1928] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 13:47:03 2022] DELETE /identity/v3/users/aed84db1720b4bb2b1d083f9a8f998f0 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:47:03.340802 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fe723e-6f99-48f2-8fac-1850184a933a None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:03.340949 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028045d8-6b76-44b2-b4c2-6b64d0dd53f2 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:03.341205 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028045d8-6b76-44b2-b4c2-6b64d0dd53f2 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:03.341335 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fe723e-6f99-48f2-8fac-1850184a933a None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:03.341579 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fe723e-6f99-48f2-8fac-1850184a933a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:03.341682 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028045d8-6b76-44b2-b4c2-6b64d0dd53f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:03.365193 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-028045d8-6b76-44b2-b4c2-6b64d0dd53f2 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:47:03.366440 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-028045d8-6b76-44b2-b4c2-6b64d0dd53f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:03.383920 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:47:03.396153 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-e5fe723e-6f99-48f2-8fac-1850184a933a None None] MFA Rules not processed for user `f0284f920ee849e4a95b32626395096c`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:47:03.397765 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-e5fe723e-6f99-48f2-8fac-1850184a933a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:03.436106 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 958/1929] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:47:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 13:47:03.437603 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028045d8-6b76-44b2-b4c2-6b64d0dd53f2 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:03.438981 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-028045d8-6b76-44b2-b4c2-6b64d0dd53f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:03.453602 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028045d8-6b76-44b2-b4c2-6b64d0dd53f2 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:03.462868 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463c8672-18a5-43df-a023-62d2e76e67c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:03.463536 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463c8672-18a5-43df-a023-62d2e76e67c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:03.463823 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463c8672-18a5-43df-a023-62d2e76e67c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:03.464667 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463c8672-18a5-43df-a023-62d2e76e67c9 admin admin] PATH_INFO: `/v3/projects/12536ebe236543c8825d177f03048642` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:03.465902 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463c8672-18a5-43df-a023-62d2e76e67c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12536ebe236543c8825d177f03048642)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:03.467939 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463c8672-18a5-43df-a023-62d2e76e67c9 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:03.480941 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 972/1930] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:47:03 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 13:47:03.481936 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fe723e-6f99-48f2-8fac-1850184a933a None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:03.483330 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-e5fe723e-6f99-48f2-8fac-1850184a933a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:03.500629 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:47:03.543567 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-463c8672-18a5-43df-a023-62d2e76e67c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12536ebe236543c8825d177f03048642', 'request_id': 'req-463c8672-18a5-43df-a023-62d2e76e67c9'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:03.550662 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-463c8672-18a5-43df-a023-62d2e76e67c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12536ebe236543c8825d177f03048642', 'request_id': 'req-463c8672-18a5-43df-a023-62d2e76e67c9'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:03.557508 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-463c8672-18a5-43df-a023-62d2e76e67c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12536ebe236543c8825d177f03048642', 'request_id': 'req-463c8672-18a5-43df-a023-62d2e76e67c9'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:03.580272 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-463c8672-18a5-43df-a023-62d2e76e67c9 admin admin] The token cache is being invalidate because project 12536ebe236543c8825d177f03048642 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 13:47:03.580572 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-463c8672-18a5-43df-a023-62d2e76e67c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:03.581637 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-463c8672-18a5-43df-a023-62d2e76e67c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:03.586698 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 959/1931] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 13:47:03 2022] DELETE /identity/v3/projects/12536ebe236543c8825d177f03048642 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:47:03.587710 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463c8672-18a5-43df-a023-62d2e76e67c9 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:03.589707 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-463c8672-18a5-43df-a023-62d2e76e67c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:03.604311 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fe723e-6f99-48f2-8fac-1850184a933a None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:03.667897 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463c8672-18a5-43df-a023-62d2e76e67c9 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:03.669915 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84cfb47-6439-4c36-91aa-138ec0019279 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:03.670562 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84cfb47-6439-4c36-91aa-138ec0019279 service neutron] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:03.670915 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84cfb47-6439-4c36-91aa-138ec0019279 service neutron] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:03.671202 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84cfb47-6439-4c36-91aa-138ec0019279 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:03.673426 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-a84cfb47-6439-4c36-91aa-138ec0019279 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:03.674136 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd124daf-263a-4dd4-bc90-2979be00d3b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:03.674877 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd124daf-263a-4dd4-bc90-2979be00d3b6 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:03.675163 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd124daf-263a-4dd4-bc90-2979be00d3b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:03.675537 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd124daf-263a-4dd4-bc90-2979be00d3b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:03.676840 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd124daf-263a-4dd4-bc90-2979be00d3b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:03.679395 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd124daf-263a-4dd4-bc90-2979be00d3b6 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:03.690189 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84cfb47-6439-4c36-91aa-138ec0019279 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:03.691790 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84cfb47-6439-4c36-91aa-138ec0019279 service neutron] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:03.705677 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 973/1932] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:47:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:47:03.706601 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84cfb47-6439-4c36-91aa-138ec0019279 service neutron] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:03.710483 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-a84cfb47-6439-4c36-91aa-138ec0019279 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:03.715612 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 960/1933] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 13:47:03 2022] POST /identity/v3/projects => generated 381 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:47:03.720539 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd124daf-263a-4dd4-bc90-2979be00d3b6 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:03.732692 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84cfb47-6439-4c36-91aa-138ec0019279 service neutron] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:03.802245 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:47:03.848180 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd124daf-263a-4dd4-bc90-2979be00d3b6 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:03.856716 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d32cfc-2a9d-485e-b6e5-6fa78823cd35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:03.857529 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d32cfc-2a9d-485e-b6e5-6fa78823cd35 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:03.857987 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d32cfc-2a9d-485e-b6e5-6fa78823cd35 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:03.858267 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d32cfc-2a9d-485e-b6e5-6fa78823cd35 admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:03.859034 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d32cfc-2a9d-485e-b6e5-6fa78823cd35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:03.862328 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d32cfc-2a9d-485e-b6e5-6fa78823cd35 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:03.867635 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3f2f04-2acb-48ed-b1d3-75d0f49f5846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:03.868309 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3f2f04-2acb-48ed-b1d3-75d0f49f5846 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:03.868600 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3f2f04-2acb-48ed-b1d3-75d0f49f5846 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:03.868877 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3f2f04-2acb-48ed-b1d3-75d0f49f5846 admin admin] PATH_INFO: `/v3/users/1e0e0faff01640ac9bf4763b615b5204` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:03.873513 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3f2f04-2acb-48ed-b1d3-75d0f49f5846 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e0e0faff01640ac9bf4763b615b5204)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:03.875817 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3f2f04-2acb-48ed-b1d3-75d0f49f5846 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:03.895943 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 961/1934] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 13:47:03 2022] POST /identity/v3/users => generated 291 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 13:47:03.902602 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d32cfc-2a9d-485e-b6e5-6fa78823cd35 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:03.937367 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d32cfc-2a9d-485e-b6e5-6fa78823cd35 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:03.951417 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25239f27-5cb2-45b8-8926-d1ff25d13c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:03.952304 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25239f27-5cb2-45b8-8926-d1ff25d13c97 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:03.954557 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25239f27-5cb2-45b8-8926-d1ff25d13c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:03.954850 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25239f27-5cb2-45b8-8926-d1ff25d13c97 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:03.955577 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25239f27-5cb2-45b8-8926-d1ff25d13c97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:03.960365 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25239f27-5cb2-45b8-8926-d1ff25d13c97 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:03.987367 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-8e3f2f04-2acb-48ed-b1d3-75d0f49f5846 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e0e0faff01640ac9bf4763b615b5204'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:03.994640 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 962/1935] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:47:03 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:47:03.999422 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25239f27-5cb2-45b8-8926-d1ff25d13c97 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:04.001186 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-8e3f2f04-2acb-48ed-b1d3-75d0f49f5846 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e0e0faff01640ac9bf4763b615b5204'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:04.016721 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-8e3f2f04-2acb-48ed-b1d3-75d0f49f5846 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e0e0faff01640ac9bf4763b615b5204'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:04.017529 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-8e3f2f04-2acb-48ed-b1d3-75d0f49f5846 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e0e0faff01640ac9bf4763b615b5204'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:04.018620 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-8e3f2f04-2acb-48ed-b1d3-75d0f49f5846 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e0e0faff01640ac9bf4763b615b5204'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:04.029088 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:47:04.042212 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25239f27-5cb2-45b8-8926-d1ff25d13c97 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:04.069043 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 974/1936] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 13:47:03 2022] DELETE /identity/v3/users/1e0e0faff01640ac9bf4763b615b5204 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:47:04.076570 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d462c57a-c287-44d0-a183-5c3d58236e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:04.077279 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d462c57a-c287-44d0-a183-5c3d58236e0c admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:04.077600 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d462c57a-c287-44d0-a183-5c3d58236e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:04.078147 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d462c57a-c287-44d0-a183-5c3d58236e0c admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:04.078730 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d462c57a-c287-44d0-a183-5c3d58236e0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:04.080332 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d462c57a-c287-44d0-a183-5c3d58236e0c admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:04.115588 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 963/1937] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:47:03 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:47:04.120303 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3f2f04-2acb-48ed-b1d3-75d0f49f5846 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:04.122595 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-8e3f2f04-2acb-48ed-b1d3-75d0f49f5846 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:04.253902 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3f2f04-2acb-48ed-b1d3-75d0f49f5846 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:04.271688 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329095f1-fe8c-4eca-9ad1-7daf696bc525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:04.272936 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329095f1-fe8c-4eca-9ad1-7daf696bc525 admin admin] REQUEST_METHOD: `PUT` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:04.273236 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329095f1-fe8c-4eca-9ad1-7daf696bc525 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:04.273541 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329095f1-fe8c-4eca-9ad1-7daf696bc525 admin admin] PATH_INFO: `/v3/projects/4ff1be3204ed49a09fa5012107116be0/users/377822fc1b6b41689428a60b16d2d9e4/roles/759f389b221049c4b01faf3b984cd8d7` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:04.304319 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329095f1-fe8c-4eca-9ad1-7daf696bc525 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ff1be3204ed49a09fa5012107116be0, user_id=377822fc1b6b41689428a60b16d2d9e4, role_id=759f389b221049c4b01faf3b984cd8d7)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:04.307047 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329095f1-fe8c-4eca-9ad1-7daf696bc525 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:04.360747 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbfe6a5-31c5-4593-8924-cf7d81c7531b None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:04.361039 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbfe6a5-31c5-4593-8924-cf7d81c7531b None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:04.361318 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbfe6a5-31c5-4593-8924-cf7d81c7531b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:04.365675 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 975/1938] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 13:47:04 2022] PUT /identity/v3/projects/4ff1be3204ed49a09fa5012107116be0/users/377822fc1b6b41689428a60b16d2d9e4/roles/759f389b221049c4b01faf3b984cd8d7 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:47:04.413238 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-0cbfe6a5-31c5-4593-8924-cf7d81c7531b None None] MFA Rules not processed for user `377822fc1b6b41689428a60b16d2d9e4`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:47:04.414603 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-0cbfe6a5-31c5-4593-8924-cf7d81c7531b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:04.541244 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329095f1-fe8c-4eca-9ad1-7daf696bc525 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:04.543194 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-329095f1-fe8c-4eca-9ad1-7daf696bc525 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:04.556922 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 964/1939] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:47:04 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:47:04.557905 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbfe6a5-31c5-4593-8924-cf7d81c7531b None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:04.652168 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329095f1-fe8c-4eca-9ad1-7daf696bc525 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:04.655913 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbfe6a5-31c5-4593-8924-cf7d81c7531b None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:04.658232 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5fdb80-0afb-46ad-aa73-aae51fd14213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:04.658858 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5fdb80-0afb-46ad-aa73-aae51fd14213 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:04.659200 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5fdb80-0afb-46ad-aa73-aae51fd14213 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:04.659486 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5fdb80-0afb-46ad-aa73-aae51fd14213 admin admin] PATH_INFO: `/v3/projects/f50e691cd38341c5a9e1251871ac4fdb` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:04.661169 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5fdb80-0afb-46ad-aa73-aae51fd14213 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f50e691cd38341c5a9e1251871ac4fdb)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:04.661345 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16f5569-de1b-4cb8-a922-984a66a74e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:04.662009 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16f5569-de1b-4cb8-a922-984a66a74e6e admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:04.662291 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16f5569-de1b-4cb8-a922-984a66a74e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:04.662567 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16f5569-de1b-4cb8-a922-984a66a74e6e admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:04.663212 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5fdb80-0afb-46ad-aa73-aae51fd14213 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:04.663411 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16f5569-de1b-4cb8-a922-984a66a74e6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:04.665578 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16f5569-de1b-4cb8-a922-984a66a74e6e admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:04.733583 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 965/1940] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 13:47:04 2022] POST /identity/v3/projects => generated 385 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:47:04.737096 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16f5569-de1b-4cb8-a922-984a66a74e6e admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:04.758295 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:47:04.777226 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-1e5fdb80-0afb-46ad-aa73-aae51fd14213 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f50e691cd38341c5a9e1251871ac4fdb', 'request_id': 'req-1e5fdb80-0afb-46ad-aa73-aae51fd14213'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:04.798019 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-1e5fdb80-0afb-46ad-aa73-aae51fd14213 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f50e691cd38341c5a9e1251871ac4fdb', 'request_id': 'req-1e5fdb80-0afb-46ad-aa73-aae51fd14213'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:04.808314 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-1e5fdb80-0afb-46ad-aa73-aae51fd14213 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f50e691cd38341c5a9e1251871ac4fdb', 'request_id': 'req-1e5fdb80-0afb-46ad-aa73-aae51fd14213'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:04.820091 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:47:04.829683 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-1e5fdb80-0afb-46ad-aa73-aae51fd14213 admin admin] The token cache is being invalidate because project f50e691cd38341c5a9e1251871ac4fdb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86933) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 13:47:04.830544 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-1e5fdb80-0afb-46ad-aa73-aae51fd14213 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:04.831551 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-1e5fdb80-0afb-46ad-aa73-aae51fd14213 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:04.837046 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 976/1941] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 13:47:04 2022] DELETE /identity/v3/projects/f50e691cd38341c5a9e1251871ac4fdb => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:47:04.838124 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5fdb80-0afb-46ad-aa73-aae51fd14213 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:04.838713 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16f5569-de1b-4cb8-a922-984a66a74e6e admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:04.841033 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-1e5fdb80-0afb-46ad-aa73-aae51fd14213 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:04.904118 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21118a99-fbd1-4bae-bb90-4897fece110f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:04.904858 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21118a99-fbd1-4bae-bb90-4897fece110f admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:04.905501 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21118a99-fbd1-4bae-bb90-4897fece110f admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:04.906377 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21118a99-fbd1-4bae-bb90-4897fece110f admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:04.907919 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21118a99-fbd1-4bae-bb90-4897fece110f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:04.909426 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21118a99-fbd1-4bae-bb90-4897fece110f admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:04.939260 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5fdb80-0afb-46ad-aa73-aae51fd14213 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:04.940320 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 966/1942] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 13:47:04 2022] POST /identity/v3/users => generated 299 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:47:04.944731 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21118a99-fbd1-4bae-bb90-4897fece110f admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:04.944931 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af792bc8-4089-496d-ab54-c3e8bb43eab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:04.945581 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af792bc8-4089-496d-ab54-c3e8bb43eab5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:04.945865 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af792bc8-4089-496d-ab54-c3e8bb43eab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:04.946140 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af792bc8-4089-496d-ab54-c3e8bb43eab5 admin admin] PATH_INFO: `/v3/users/b54341f7a6ab4450b53924664cb4732a` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:04.947343 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af792bc8-4089-496d-ab54-c3e8bb43eab5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b54341f7a6ab4450b53924664cb4732a)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:04.948005 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-21118a99-fbd1-4bae-bb90-4897fece110f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:04.949259 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af792bc8-4089-496d-ab54-c3e8bb43eab5 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:04.964689 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21118a99-fbd1-4bae-bb90-4897fece110f admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:04.970497 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682506f4-e7f6-451f-970f-4db38c3a48b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:04.970757 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682506f4-e7f6-451f-970f-4db38c3a48b4 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:04.970982 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682506f4-e7f6-451f-970f-4db38c3a48b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:04.971262 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682506f4-e7f6-451f-970f-4db38c3a48b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:04.971962 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682506f4-e7f6-451f-970f-4db38c3a48b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:04.973614 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682506f4-e7f6-451f-970f-4db38c3a48b4 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:04.985539 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 967/1943] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:47:04 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 13:47:04.989938 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682506f4-e7f6-451f-970f-4db38c3a48b4 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:05.000677 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682506f4-e7f6-451f-970f-4db38c3a48b4 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:05.006104 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9b58d7-b4f6-4231-aad3-44637510d8eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:05.006345 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9b58d7-b4f6-4231-aad3-44637510d8eb admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:05.006954 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9b58d7-b4f6-4231-aad3-44637510d8eb admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:05.006954 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9b58d7-b4f6-4231-aad3-44637510d8eb admin admin] PATH_INFO: `/v3/projects/5841eeababae4d9882f7a0680b875208/users/d5a4b512f7f44f22a7067bd8be76da35/roles/6e0201a9a6484f7d8b112b1eafff70bd` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:05.009083 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-af792bc8-4089-496d-ab54-c3e8bb43eab5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b54341f7a6ab4450b53924664cb4732a'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:05.018351 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-af792bc8-4089-496d-ab54-c3e8bb43eab5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b54341f7a6ab4450b53924664cb4732a'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:05.025867 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9b58d7-b4f6-4231-aad3-44637510d8eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=5841eeababae4d9882f7a0680b875208, user_id=d5a4b512f7f44f22a7067bd8be76da35, role_id=6e0201a9a6484f7d8b112b1eafff70bd)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:05.027535 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-af792bc8-4089-496d-ab54-c3e8bb43eab5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b54341f7a6ab4450b53924664cb4732a'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:05.028433 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-af792bc8-4089-496d-ab54-c3e8bb43eab5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b54341f7a6ab4450b53924664cb4732a'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:05.029027 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9b58d7-b4f6-4231-aad3-44637510d8eb admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:05.029131 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-af792bc8-4089-496d-ab54-c3e8bb43eab5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b54341f7a6ab4450b53924664cb4732a'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:05.061658 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 968/1944] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 13:47:04 2022] PUT /identity/v3/projects/5841eeababae4d9882f7a0680b875208/users/d5a4b512f7f44f22a7067bd8be76da35/roles/6e0201a9a6484f7d8b112b1eafff70bd => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 13:47:05.061999 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 977/1945] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 13:47:04 2022] DELETE /identity/v3/users/b54341f7a6ab4450b53924664cb4732a => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:47:05.063104 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9b58d7-b4f6-4231-aad3-44637510d8eb admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:05.065271 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-ee9b58d7-b4f6-4231-aad3-44637510d8eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:05.164409 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9b58d7-b4f6-4231-aad3-44637510d8eb admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:05.167353 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af792bc8-4089-496d-ab54-c3e8bb43eab5 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:05.168863 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-af792bc8-4089-496d-ab54-c3e8bb43eab5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:05.194952 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:47:05.249386 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af792bc8-4089-496d-ab54-c3e8bb43eab5 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:05.251241 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada7b055-4c55-4d8b-ba19-37316786f1d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:05.251925 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada7b055-4c55-4d8b-ba19-37316786f1d8 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:05.252159 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada7b055-4c55-4d8b-ba19-37316786f1d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:05.252441 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada7b055-4c55-4d8b-ba19-37316786f1d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:05.254047 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada7b055-4c55-4d8b-ba19-37316786f1d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:05.255551 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada7b055-4c55-4d8b-ba19-37316786f1d8 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:05.257619 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d71ca7-1957-4c35-9448-d182c2730637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:05.258388 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d71ca7-1957-4c35-9448-d182c2730637 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:05.258860 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d71ca7-1957-4c35-9448-d182c2730637 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:05.259235 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d71ca7-1957-4c35-9448-d182c2730637 admin admin] PATH_INFO: `/v3/projects/0431e54a61d44cf3bfeaa6681788c337` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:05.260694 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d71ca7-1957-4c35-9448-d182c2730637 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0431e54a61d44cf3bfeaa6681788c337)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:05.262936 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d71ca7-1957-4c35-9448-d182c2730637 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:05.267833 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 969/1946] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:47:05 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:47:05.272189 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada7b055-4c55-4d8b-ba19-37316786f1d8 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:05.283313 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada7b055-4c55-4d8b-ba19-37316786f1d8 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:05.288166 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c784b11-b68e-45b7-8fba-348d8e9fb269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:05.288839 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c784b11-b68e-45b7-8fba-348d8e9fb269 admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:05.289124 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c784b11-b68e-45b7-8fba-348d8e9fb269 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:05.289402 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c784b11-b68e-45b7-8fba-348d8e9fb269 admin admin] PATH_INFO: `/v3/projects/5841eeababae4d9882f7a0680b875208/users/d5a4b512f7f44f22a7067bd8be76da35/roles/6e0201a9a6484f7d8b112b1eafff70bd` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:05.290600 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:47:05.291900 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c784b11-b68e-45b7-8fba-348d8e9fb269 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5841eeababae4d9882f7a0680b875208, user_id=d5a4b512f7f44f22a7067bd8be76da35, role_id=6e0201a9a6484f7d8b112b1eafff70bd)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:05.294659 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c784b11-b68e-45b7-8fba-348d8e9fb269 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:05.309527 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 970/1947] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 13:47:05 2022] PUT /identity/v3/projects/5841eeababae4d9882f7a0680b875208/users/d5a4b512f7f44f22a7067bd8be76da35/roles/6e0201a9a6484f7d8b112b1eafff70bd => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:47:05.323275 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008e9fda-8496-4fdc-a52b-cd70ec7676da None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:05.323729 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008e9fda-8496-4fdc-a52b-cd70ec7676da None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:05.324116 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008e9fda-8496-4fdc-a52b-cd70ec7676da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:05.329304 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-56d71ca7-1957-4c35-9448-d182c2730637 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0431e54a61d44cf3bfeaa6681788c337', 'request_id': 'req-56d71ca7-1957-4c35-9448-d182c2730637'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:05.339669 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-56d71ca7-1957-4c35-9448-d182c2730637 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0431e54a61d44cf3bfeaa6681788c337', 'request_id': 'req-56d71ca7-1957-4c35-9448-d182c2730637'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:05.348094 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-56d71ca7-1957-4c35-9448-d182c2730637 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0431e54a61d44cf3bfeaa6681788c337', 'request_id': 'req-56d71ca7-1957-4c35-9448-d182c2730637'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:05.362343 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-008e9fda-8496-4fdc-a52b-cd70ec7676da None None] MFA Rules not processed for user `d5a4b512f7f44f22a7067bd8be76da35`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:47:05.362940 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-56d71ca7-1957-4c35-9448-d182c2730637 admin admin] The token cache is being invalidate because project 0431e54a61d44cf3bfeaa6681788c337 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86933) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 13:47:05.363330 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-56d71ca7-1957-4c35-9448-d182c2730637 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:05.363997 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-008e9fda-8496-4fdc-a52b-cd70ec7676da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:05.364188 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-56d71ca7-1957-4c35-9448-d182c2730637 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:05.365972 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 978/1948] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 13:47:05 2022] DELETE /identity/v3/projects/0431e54a61d44cf3bfeaa6681788c337 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 13:47:05.370715 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb335971-4f19-42c3-bfb2-e6dc66cf89b3 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:05.371373 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb335971-4f19-42c3-bfb2-e6dc66cf89b3 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:05.371373 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb335971-4f19-42c3-bfb2-e6dc66cf89b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:05.394050 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-cb335971-4f19-42c3-bfb2-e6dc66cf89b3 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:47:05.395279 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-cb335971-4f19-42c3-bfb2-e6dc66cf89b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:05.458490 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 971/1949] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:47:05 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:47:05.459418 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008e9fda-8496-4fdc-a52b-cd70ec7676da None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:05.460827 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-008e9fda-8496-4fdc-a52b-cd70ec7676da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:05.469727 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 979/1950] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:47:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 13:47:05.470150 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb335971-4f19-42c3-bfb2-e6dc66cf89b3 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:05.473355 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-cb335971-4f19-42c3-bfb2-e6dc66cf89b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:05.475577 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008e9fda-8496-4fdc-a52b-cd70ec7676da None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:05.480179 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19f591a-f9ec-4cfd-8c53-a6fce202093f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:05.480891 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19f591a-f9ec-4cfd-8c53-a6fce202093f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:05.481261 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19f591a-f9ec-4cfd-8c53-a6fce202093f admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:05.481703 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19f591a-f9ec-4cfd-8c53-a6fce202093f admin admin] PATH_INFO: `/v3/projects/dfd7e70158a140e9a8f257e76670f756` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:05.482922 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19f591a-f9ec-4cfd-8c53-a6fce202093f admin admin] RBAC: Authorizing `identity:delete_project(project_id=dfd7e70158a140e9a8f257e76670f756)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:05.484880 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19f591a-f9ec-4cfd-8c53-a6fce202093f admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:05.533146 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb335971-4f19-42c3-bfb2-e6dc66cf89b3 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:05.537674 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-e19f591a-f9ec-4cfd-8c53-a6fce202093f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dfd7e70158a140e9a8f257e76670f756', 'request_id': 'req-e19f591a-f9ec-4cfd-8c53-a6fce202093f'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:05.544617 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-e19f591a-f9ec-4cfd-8c53-a6fce202093f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dfd7e70158a140e9a8f257e76670f756', 'request_id': 'req-e19f591a-f9ec-4cfd-8c53-a6fce202093f'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:05.555148 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-e19f591a-f9ec-4cfd-8c53-a6fce202093f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dfd7e70158a140e9a8f257e76670f756', 'request_id': 'req-e19f591a-f9ec-4cfd-8c53-a6fce202093f'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:05.569694 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-e19f591a-f9ec-4cfd-8c53-a6fce202093f admin admin] The token cache is being invalidate because project dfd7e70158a140e9a8f257e76670f756 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 13:47:05.569987 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-e19f591a-f9ec-4cfd-8c53-a6fce202093f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:05.570642 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-e19f591a-f9ec-4cfd-8c53-a6fce202093f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:05.572196 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 972/1951] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 13:47:05 2022] DELETE /identity/v3/projects/dfd7e70158a140e9a8f257e76670f756 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:47:05.575402 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3178ceb8-1012-4e32-9286-877ec21bfd39 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:05.575812 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3178ceb8-1012-4e32-9286-877ec21bfd39 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:05.576122 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3178ceb8-1012-4e32-9286-877ec21bfd39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:05.585915 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bef4899-155f-42ff-817d-63dbd174a903 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:05.586541 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bef4899-155f-42ff-817d-63dbd174a903 service cinder] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:05.586822 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bef4899-155f-42ff-817d-63dbd174a903 service cinder] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:05.587095 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bef4899-155f-42ff-817d-63dbd174a903 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:05.588845 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-9bef4899-155f-42ff-817d-63dbd174a903 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:05.596364 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-3178ceb8-1012-4e32-9286-877ec21bfd39 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:47:05.597608 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-3178ceb8-1012-4e32-9286-877ec21bfd39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:05.606594 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:47:05.607432 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:47:05.655982 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bef4899-155f-42ff-817d-63dbd174a903 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:05.657409 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bef4899-155f-42ff-817d-63dbd174a903 service cinder] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:05.663980 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 973/1952] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:47:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:47:05.665180 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24d7924-f1fd-40fd-8e72-13a8ef237fae None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:05.665466 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24d7924-f1fd-40fd-8e72-13a8ef237fae None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:05.665769 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24d7924-f1fd-40fd-8e72-13a8ef237fae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:05.666791 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 980/1953] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:47:05 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 13:47:05.669236 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bef4899-155f-42ff-817d-63dbd174a903 service cinder] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:05.679131 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bef4899-155f-42ff-817d-63dbd174a903 service cinder] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:05.684020 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586f2e77-b8af-441e-8d95-184b64588264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:05.684629 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586f2e77-b8af-441e-8d95-184b64588264 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:05.684908 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586f2e77-b8af-441e-8d95-184b64588264 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:05.685180 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586f2e77-b8af-441e-8d95-184b64588264 admin admin] PATH_INFO: `/v3/domains` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:05.685899 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586f2e77-b8af-441e-8d95-184b64588264 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:47:05.686200 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586f2e77-b8af-441e-8d95-184b64588264 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:05.688448 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-d24d7924-f1fd-40fd-8e72-13a8ef237fae None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:47:05.689894 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586f2e77-b8af-441e-8d95-184b64588264 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:05.690005 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-d24d7924-f1fd-40fd-8e72-13a8ef237fae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:05.698316 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 974/1954] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:47:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:47:05.707588 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 981/1955] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Jul 26 13:47:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:47:05.707769 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab705f3-fa21-443f-a7f5-17c75b4e503c None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:05.707769 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab705f3-fa21-443f-a7f5-17c75b4e503c None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:05.707769 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab705f3-fa21-443f-a7f5-17c75b4e503c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:05.708358 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586f2e77-b8af-441e-8d95-184b64588264 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:05.720385 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586f2e77-b8af-441e-8d95-184b64588264 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:05.726579 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6dfa45-f098-4a1b-ad07-155febf51d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:05.727460 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6dfa45-f098-4a1b-ad07-155febf51d43 admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:05.727750 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6dfa45-f098-4a1b-ad07-155febf51d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:05.728026 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6dfa45-f098-4a1b-ad07-155febf51d43 admin admin] PATH_INFO: `/v3/projects` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:05.728825 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6dfa45-f098-4a1b-ad07-155febf51d43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:05.730767 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6dfa45-f098-4a1b-ad07-155febf51d43 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:05.735014 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-1ab705f3-fa21-443f-a7f5-17c75b4e503c None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:47:05.736710 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-1ab705f3-fa21-443f-a7f5-17c75b4e503c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:05.745824 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 975/1956] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:47:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 13:47:05.750497 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab705f3-fa21-443f-a7f5-17c75b4e503c None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:05.763164 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab705f3-fa21-443f-a7f5-17c75b4e503c None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:05.770499 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a16a38-cc81-449d-8468-46dd359f3311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:05.771583 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a16a38-cc81-449d-8468-46dd359f3311 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:05.771991 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a16a38-cc81-449d-8468-46dd359f3311 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:05.772396 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a16a38-cc81-449d-8468-46dd359f3311 admin admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:05.773376 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a16a38-cc81-449d-8468-46dd359f3311 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:47:05.773828 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a16a38-cc81-449d-8468-46dd359f3311 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:05.775727 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a16a38-cc81-449d-8468-46dd359f3311 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:05.776216 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 982/1957] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 13:47:05 2022] POST /identity/v3/projects => generated 379 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 13:47:05.780531 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6dfa45-f098-4a1b-ad07-155febf51d43 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:05.799814 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 976/1958] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Jul 26 13:47:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:47:05.802295 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a16a38-cc81-449d-8468-46dd359f3311 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:05.828594 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a16a38-cc81-449d-8468-46dd359f3311 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:05.889015 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6dfa45-f098-4a1b-ad07-155febf51d43 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:05.895043 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6b6680-5bdb-4e6f-8d47-30c9f52d698a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:05.895724 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6b6680-5bdb-4e6f-8d47-30c9f52d698a admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:05.896001 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6b6680-5bdb-4e6f-8d47-30c9f52d698a admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:05.896280 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6b6680-5bdb-4e6f-8d47-30c9f52d698a admin admin] PATH_INFO: `/v3/users` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:05.896975 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6b6680-5bdb-4e6f-8d47-30c9f52d698a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:05.898928 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6b6680-5bdb-4e6f-8d47-30c9f52d698a admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:05.909380 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2013fffc-6418-47ae-847f-53488f30a591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:05.911911 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2013fffc-6418-47ae-847f-53488f30a591 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:05.912370 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2013fffc-6418-47ae-847f-53488f30a591 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:05.913033 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2013fffc-6418-47ae-847f-53488f30a591 admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:05.914200 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2013fffc-6418-47ae-847f-53488f30a591 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:05.916576 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2013fffc-6418-47ae-847f-53488f30a591 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:05.931675 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 983/1959] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 13:47:05 2022] POST /identity/v3/users => generated 290 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:47:05.936021 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6b6680-5bdb-4e6f-8d47-30c9f52d698a admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:05.949714 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6b6680-5bdb-4e6f-8d47-30c9f52d698a admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:05.959513 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674e5070-52ac-4f1f-9362-4199e60f8620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:05.960161 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674e5070-52ac-4f1f-9362-4199e60f8620 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:05.961066 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674e5070-52ac-4f1f-9362-4199e60f8620 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:05.961348 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674e5070-52ac-4f1f-9362-4199e60f8620 admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:05.962542 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674e5070-52ac-4f1f-9362-4199e60f8620 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:05.964355 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674e5070-52ac-4f1f-9362-4199e60f8620 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:05.989077 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 977/1961] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 13:47:05 2022] POST /identity/v3/projects => generated 383 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:47:05.990416 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 984/1961] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:47:05 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 13:47:05.992752 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2013fffc-6418-47ae-847f-53488f30a591 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:05.994591 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674e5070-52ac-4f1f-9362-4199e60f8620 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:06.006329 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674e5070-52ac-4f1f-9362-4199e60f8620 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:06.076152 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2013fffc-6418-47ae-847f-53488f30a591 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:06.080223 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:47:06.081536 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9d128c-fbd8-4436-a9fe-63595f88a208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:06.082334 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9d128c-fbd8-4436-a9fe-63595f88a208 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:06.082727 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9d128c-fbd8-4436-a9fe-63595f88a208 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:06.082979 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9d128c-fbd8-4436-a9fe-63595f88a208 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:06.083863 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9d128c-fbd8-4436-a9fe-63595f88a208 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:06.085406 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9d128c-fbd8-4436-a9fe-63595f88a208 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:06.087690 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a147ce4d-de7f-4a6d-8779-dad6a2c5befe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:06.088406 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a147ce4d-de7f-4a6d-8779-dad6a2c5befe admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:06.089020 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a147ce4d-de7f-4a6d-8779-dad6a2c5befe admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:06.089363 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a147ce4d-de7f-4a6d-8779-dad6a2c5befe admin admin] PATH_INFO: `/v3/users` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:06.090423 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a147ce4d-de7f-4a6d-8779-dad6a2c5befe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:06.092632 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a147ce4d-de7f-4a6d-8779-dad6a2c5befe admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:06.098129 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 978/1962] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:47:05 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:47:06.101347 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9d128c-fbd8-4436-a9fe-63595f88a208 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:06.118315 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 985/1963] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 13:47:05 2022] POST /identity/v3/users => generated 292 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:47:06.118439 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9d128c-fbd8-4436-a9fe-63595f88a208 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:06.122352 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a147ce4d-de7f-4a6d-8779-dad6a2c5befe admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:06.123568 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee9c488-6da8-4378-a474-14435007290a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:06.124474 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee9c488-6da8-4378-a474-14435007290a admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:06.124764 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee9c488-6da8-4378-a474-14435007290a admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:06.125279 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee9c488-6da8-4378-a474-14435007290a admin admin] PATH_INFO: `/v3/projects/fe13e91845924ce3bd4c49e6ff7f0db1/users/6dc005c95d1f44e1afb4664d81908808/roles/759f389b221049c4b01faf3b984cd8d7` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:06.129316 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:47:06.138240 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a147ce4d-de7f-4a6d-8779-dad6a2c5befe admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:06.144669 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f5ea6a-9d58-41f3-9097-8f6197d34c46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:06.145503 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f5ea6a-9d58-41f3-9097-8f6197d34c46 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:06.145870 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f5ea6a-9d58-41f3-9097-8f6197d34c46 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:06.146253 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f5ea6a-9d58-41f3-9097-8f6197d34c46 admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:06.147665 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f5ea6a-9d58-41f3-9097-8f6197d34c46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:06.148285 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee9c488-6da8-4378-a474-14435007290a admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe13e91845924ce3bd4c49e6ff7f0db1, user_id=6dc005c95d1f44e1afb4664d81908808, role_id=759f389b221049c4b01faf3b984cd8d7)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:06.149507 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f5ea6a-9d58-41f3-9097-8f6197d34c46 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:06.151248 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee9c488-6da8-4378-a474-14435007290a admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:06.162435 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 986/1964] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:47:06 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 13:47:06.167539 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f5ea6a-9d58-41f3-9097-8f6197d34c46 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:06.168413 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 979/1965] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 13:47:06 2022] PUT /identity/v3/projects/fe13e91845924ce3bd4c49e6ff7f0db1/users/6dc005c95d1f44e1afb4664d81908808/roles/759f389b221049c4b01faf3b984cd8d7 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 13:47:06.177314 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f5ea6a-9d58-41f3-9097-8f6197d34c46 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:06.183500 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6491fb39-8e56-4acf-a6bb-8b79b426c792 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:06.186328 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6491fb39-8e56-4acf-a6bb-8b79b426c792 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:06.186927 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6491fb39-8e56-4acf-a6bb-8b79b426c792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:06.240559 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-6491fb39-8e56-4acf-a6bb-8b79b426c792 None None] MFA Rules not processed for user `6dc005c95d1f44e1afb4664d81908808`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:47:06.242467 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-6491fb39-8e56-4acf-a6bb-8b79b426c792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:06.258691 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eb0764-c4b5-46dc-86b9-78c12f7c7c93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:06.259579 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eb0764-c4b5-46dc-86b9-78c12f7c7c93 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:06.260044 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eb0764-c4b5-46dc-86b9-78c12f7c7c93 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:06.261288 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eb0764-c4b5-46dc-86b9-78c12f7c7c93 admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:06.263035 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5eb0764-c4b5-46dc-86b9-78c12f7c7c93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:06.266226 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5eb0764-c4b5-46dc-86b9-78c12f7c7c93 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:06.279255 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 987/1966] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:47:06 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 13:47:06.284074 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eb0764-c4b5-46dc-86b9-78c12f7c7c93 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:06.302701 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eb0764-c4b5-46dc-86b9-78c12f7c7c93 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:06.310501 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2254874-99c3-44f2-9472-c226dc9038f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:06.311657 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2254874-99c3-44f2-9472-c226dc9038f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:06.313033 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2254874-99c3-44f2-9472-c226dc9038f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:06.313492 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2254874-99c3-44f2-9472-c226dc9038f0 admin admin] PATH_INFO: `/v3/projects/52ce91fedeb5457c996654431ae78c69/users/00cbd308d03d480f87fb6c23273dbe22/roles/759f389b221049c4b01faf3b984cd8d7` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:06.327702 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 980/1967] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:47:06 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:47:06.334030 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6491fb39-8e56-4acf-a6bb-8b79b426c792 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:06.336629 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2254874-99c3-44f2-9472-c226dc9038f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52ce91fedeb5457c996654431ae78c69, user_id=00cbd308d03d480f87fb6c23273dbe22, role_id=759f389b221049c4b01faf3b984cd8d7)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:06.340071 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2254874-99c3-44f2-9472-c226dc9038f0 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:06.353716 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6491fb39-8e56-4acf-a6bb-8b79b426c792 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:06.357355 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 988/1968] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 13:47:06 2022] PUT /identity/v3/projects/52ce91fedeb5457c996654431ae78c69/users/00cbd308d03d480f87fb6c23273dbe22/roles/759f389b221049c4b01faf3b984cd8d7 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:47:06.375227 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0835b687-68cd-44a9-beb2-ce3c87321d19 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:06.377537 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0835b687-68cd-44a9-beb2-ce3c87321d19 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:06.377870 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0835b687-68cd-44a9-beb2-ce3c87321d19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:06.429973 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-0835b687-68cd-44a9-beb2-ce3c87321d19 None None] MFA Rules not processed for user `00cbd308d03d480f87fb6c23273dbe22`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:47:06.431235 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-0835b687-68cd-44a9-beb2-ce3c87321d19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:06.453803 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c28ccea-5cc7-47fe-b1f8-628967e6e9cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:06.454859 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c28ccea-5cc7-47fe-b1f8-628967e6e9cf admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:06.455494 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c28ccea-5cc7-47fe-b1f8-628967e6e9cf admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:06.455936 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c28ccea-5cc7-47fe-b1f8-628967e6e9cf admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:06.457005 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c28ccea-5cc7-47fe-b1f8-628967e6e9cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:06.458972 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c28ccea-5cc7-47fe-b1f8-628967e6e9cf admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:06.497515 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 981/1969] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 13:47:06 2022] POST /identity/v3/projects => generated 379 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:47:06.497702 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 989/1970] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:47:06 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:47:06.497702 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0835b687-68cd-44a9-beb2-ce3c87321d19 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:06.499128 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c28ccea-5cc7-47fe-b1f8-628967e6e9cf admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:06.499257 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-0835b687-68cd-44a9-beb2-ce3c87321d19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:06.540521 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:47:06.558173 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0835b687-68cd-44a9-beb2-ce3c87321d19 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:06.569354 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c28ccea-5cc7-47fe-b1f8-628967e6e9cf admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:06.574126 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de79165-e42d-4f64-b8e4-ba1d5e2c0746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:06.574751 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de79165-e42d-4f64-b8e4-ba1d5e2c0746 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:06.575035 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de79165-e42d-4f64-b8e4-ba1d5e2c0746 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:06.575312 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de79165-e42d-4f64-b8e4-ba1d5e2c0746 admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:06.575980 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de79165-e42d-4f64-b8e4-ba1d5e2c0746 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:06.578145 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de79165-e42d-4f64-b8e4-ba1d5e2c0746 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:06.586263 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:47:06.602439 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 982/1971] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 13:47:06 2022] POST /identity/v3/users => generated 296 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:47:06.603335 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de79165-e42d-4f64-b8e4-ba1d5e2c0746 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:06.605358 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-6de79165-e42d-4f64-b8e4-ba1d5e2c0746 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:06.612085 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83aa2ce-aee9-4b2c-9d51-5fdd69448a98 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:06.612718 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83aa2ce-aee9-4b2c-9d51-5fdd69448a98 service cinder] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:06.613000 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83aa2ce-aee9-4b2c-9d51-5fdd69448a98 service cinder] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:06.613278 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83aa2ce-aee9-4b2c-9d51-5fdd69448a98 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:06.615080 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-d83aa2ce-aee9-4b2c-9d51-5fdd69448a98 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:06.619209 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de79165-e42d-4f64-b8e4-ba1d5e2c0746 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:06.623558 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b941ce-01af-4567-ac45-80f0604d6407 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:06.624172 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b941ce-01af-4567-ac45-80f0604d6407 service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:06.625176 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b941ce-01af-4567-ac45-80f0604d6407 service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:06.625448 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b941ce-01af-4567-ac45-80f0604d6407 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:06.677699 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83aa2ce-aee9-4b2c-9d51-5fdd69448a98 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:06.679096 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83aa2ce-aee9-4b2c-9d51-5fdd69448a98 service cinder] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:06.682978 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b941ce-01af-4567-ac45-80f0604d6407 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:06.684396 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b941ce-01af-4567-ac45-80f0604d6407 service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:06.688817 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 990/1972] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:47:06 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:47:06.689647 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83aa2ce-aee9-4b2c-9d51-5fdd69448a98 service cinder] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:06.694193 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 983/1973] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:47:06 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:47:06.700905 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83aa2ce-aee9-4b2c-9d51-5fdd69448a98 service cinder] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:06.706445 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa971e6-644f-4e91-af89-8aa95411b7f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:06.706573 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa971e6-644f-4e91-af89-8aa95411b7f2 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:06.706800 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa971e6-644f-4e91-af89-8aa95411b7f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:06.707071 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa971e6-644f-4e91-af89-8aa95411b7f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:06.707768 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa971e6-644f-4e91-af89-8aa95411b7f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:06.709356 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa971e6-644f-4e91-af89-8aa95411b7f2 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:06.723268 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 991/1974] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:47:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:47:06.724456 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b941ce-01af-4567-ac45-80f0604d6407 service cinder] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:06.737596 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b941ce-01af-4567-ac45-80f0604d6407 service cinder] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:06.743008 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8317b66-f8cf-4cfb-b209-213d1adceb4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:06.743887 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8317b66-f8cf-4cfb-b209-213d1adceb4c admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:06.744366 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8317b66-f8cf-4cfb-b209-213d1adceb4c admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:06.744743 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8317b66-f8cf-4cfb-b209-213d1adceb4c admin admin] PATH_INFO: `/v3/projects/44f686a9aad14f99974057b16d585f8d/users/a1d2e257aa8543239d8f2e4778b4c6d8/roles/6e0201a9a6484f7d8b112b1eafff70bd` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:06.763179 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8317b66-f8cf-4cfb-b209-213d1adceb4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=44f686a9aad14f99974057b16d585f8d, user_id=a1d2e257aa8543239d8f2e4778b4c6d8, role_id=6e0201a9a6484f7d8b112b1eafff70bd)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:06.766050 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8317b66-f8cf-4cfb-b209-213d1adceb4c admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:06.781843 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 984/1975] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 13:47:06 2022] PUT /identity/v3/projects/44f686a9aad14f99974057b16d585f8d/users/a1d2e257aa8543239d8f2e4778b4c6d8/roles/6e0201a9a6484f7d8b112b1eafff70bd => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:47:06.787090 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa971e6-644f-4e91-af89-8aa95411b7f2 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:06.802902 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa971e6-644f-4e91-af89-8aa95411b7f2 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:06.878635 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0fd0f0-acfe-4cef-934a-2c842d58dcf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:06.879783 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0fd0f0-acfe-4cef-934a-2c842d58dcf9 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:06.880132 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0fd0f0-acfe-4cef-934a-2c842d58dcf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:06.880468 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0fd0f0-acfe-4cef-934a-2c842d58dcf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:06.881391 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0fd0f0-acfe-4cef-934a-2c842d58dcf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:06.883341 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0fd0f0-acfe-4cef-934a-2c842d58dcf9 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:06.895404 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 992/1976] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:47:06 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 13:47:06.899864 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8317b66-f8cf-4cfb-b209-213d1adceb4c admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:06.912558 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8317b66-f8cf-4cfb-b209-213d1adceb4c admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:06.918159 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cac0712-2902-4f4f-8652-827f53534a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:06.919002 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cac0712-2902-4f4f-8652-827f53534a5b admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:06.919299 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cac0712-2902-4f4f-8652-827f53534a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:06.919671 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cac0712-2902-4f4f-8652-827f53534a5b admin admin] PATH_INFO: `/v3/projects/44f686a9aad14f99974057b16d585f8d/users/a1d2e257aa8543239d8f2e4778b4c6d8/roles/6e0201a9a6484f7d8b112b1eafff70bd` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:06.921970 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cac0712-2902-4f4f-8652-827f53534a5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=44f686a9aad14f99974057b16d585f8d, user_id=a1d2e257aa8543239d8f2e4778b4c6d8, role_id=6e0201a9a6484f7d8b112b1eafff70bd)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:06.924974 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cac0712-2902-4f4f-8652-827f53534a5b admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:06.937268 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 985/1977] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 13:47:06 2022] PUT /identity/v3/projects/44f686a9aad14f99974057b16d585f8d/users/a1d2e257aa8543239d8f2e4778b4c6d8/roles/6e0201a9a6484f7d8b112b1eafff70bd => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:47:06.947659 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3437317b-f332-4509-afd5-bc5e3aaa0ed5 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:06.948455 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3437317b-f332-4509-afd5-bc5e3aaa0ed5 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:06.949667 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3437317b-f332-4509-afd5-bc5e3aaa0ed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:06.994679 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-3437317b-f332-4509-afd5-bc5e3aaa0ed5 None None] MFA Rules not processed for user `a1d2e257aa8543239d8f2e4778b4c6d8`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:47:06.995287 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-3437317b-f332-4509-afd5-bc5e3aaa0ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:07.028726 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:47:07.115030 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 993/1978] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:47:06 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:47:07.125993 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cac0712-2902-4f4f-8652-827f53534a5b admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:07.204026 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cac0712-2902-4f4f-8652-827f53534a5b admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:07.208714 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c6a689-fe53-47d6-bfa6-466d68d2ade7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:07.209843 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c6a689-fe53-47d6-bfa6-466d68d2ade7 service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:07.210223 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c6a689-fe53-47d6-bfa6-466d68d2ade7 service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:07.211623 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c6a689-fe53-47d6-bfa6-466d68d2ade7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:07.222231 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c6a689-fe53-47d6-bfa6-466d68d2ade7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:07.224193 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c6a689-fe53-47d6-bfa6-466d68d2ade7 service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:07.235540 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 986/1979] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:47:07 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:47:07.679859 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3437317b-f332-4509-afd5-bc5e3aaa0ed5 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:07.742886 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3437317b-f332-4509-afd5-bc5e3aaa0ed5 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:07.747402 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63bd252-c6e2-49e5-8ebc-9b34135d77ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:07.748053 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63bd252-c6e2-49e5-8ebc-9b34135d77ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:07.748333 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63bd252-c6e2-49e5-8ebc-9b34135d77ae admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:07.748609 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63bd252-c6e2-49e5-8ebc-9b34135d77ae admin admin] PATH_INFO: `/v3/users/6dc005c95d1f44e1afb4664d81908808` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:07.749826 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63bd252-c6e2-49e5-8ebc-9b34135d77ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=6dc005c95d1f44e1afb4664d81908808)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:07.751733 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63bd252-c6e2-49e5-8ebc-9b34135d77ae admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:07.800982 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-a63bd252-c6e2-49e5-8ebc-9b34135d77ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6dc005c95d1f44e1afb4664d81908808'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:07.809724 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-a63bd252-c6e2-49e5-8ebc-9b34135d77ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6dc005c95d1f44e1afb4664d81908808'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:07.817624 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-a63bd252-c6e2-49e5-8ebc-9b34135d77ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6dc005c95d1f44e1afb4664d81908808'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:07.818388 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-a63bd252-c6e2-49e5-8ebc-9b34135d77ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6dc005c95d1f44e1afb4664d81908808'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:07.819004 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-a63bd252-c6e2-49e5-8ebc-9b34135d77ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6dc005c95d1f44e1afb4664d81908808'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:07.837467 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 994/1980] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 13:47:07 2022] DELETE /identity/v3/users/6dc005c95d1f44e1afb4664d81908808 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:47:07.841988 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f0d26d-c69c-4392-b286-a5bb7c6437a6 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:07.842361 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f0d26d-c69c-4392-b286-a5bb7c6437a6 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:07.842704 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f0d26d-c69c-4392-b286-a5bb7c6437a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:07.861193 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-a6f0d26d-c69c-4392-b286-a5bb7c6437a6 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:47:07.862502 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-a6f0d26d-c69c-4392-b286-a5bb7c6437a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:07.924698 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:47:07.938587 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 987/1981] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:47:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:47:07.948088 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63bd252-c6e2-49e5-8ebc-9b34135d77ae admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:07.949557 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-a63bd252-c6e2-49e5-8ebc-9b34135d77ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:07.990267 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:47:08.006891 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63bd252-c6e2-49e5-8ebc-9b34135d77ae admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:08.010676 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74630c4b-dd43-4e23-8f1e-130dc5222a0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:08.011305 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74630c4b-dd43-4e23-8f1e-130dc5222a0e service neutron] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:08.011838 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74630c4b-dd43-4e23-8f1e-130dc5222a0e service neutron] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:08.011838 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74630c4b-dd43-4e23-8f1e-130dc5222a0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:08.021891 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74630c4b-dd43-4e23-8f1e-130dc5222a0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:08.023463 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74630c4b-dd43-4e23-8f1e-130dc5222a0e service neutron] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:08.038573 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 995/1982] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:47:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:47:08.365096 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f0d26d-c69c-4392-b286-a5bb7c6437a6 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:08.366229 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-a6f0d26d-c69c-4392-b286-a5bb7c6437a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:08.379045 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f0d26d-c69c-4392-b286-a5bb7c6437a6 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:08.394017 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cdb946-a595-4d4c-8f34-2df0a0cc0ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:08.394843 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cdb946-a595-4d4c-8f34-2df0a0cc0ee6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:08.395203 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cdb946-a595-4d4c-8f34-2df0a0cc0ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:08.395607 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cdb946-a595-4d4c-8f34-2df0a0cc0ee6 admin admin] PATH_INFO: `/v3/projects/fe13e91845924ce3bd4c49e6ff7f0db1` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:08.397094 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cdb946-a595-4d4c-8f34-2df0a0cc0ee6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe13e91845924ce3bd4c49e6ff7f0db1)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:08.399052 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cdb946-a595-4d4c-8f34-2df0a0cc0ee6 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:08.462560 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-39cdb946-a595-4d4c-8f34-2df0a0cc0ee6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe13e91845924ce3bd4c49e6ff7f0db1', 'request_id': 'req-39cdb946-a595-4d4c-8f34-2df0a0cc0ee6'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:08.468921 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-39cdb946-a595-4d4c-8f34-2df0a0cc0ee6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe13e91845924ce3bd4c49e6ff7f0db1', 'request_id': 'req-39cdb946-a595-4d4c-8f34-2df0a0cc0ee6'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:08.475844 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-39cdb946-a595-4d4c-8f34-2df0a0cc0ee6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe13e91845924ce3bd4c49e6ff7f0db1', 'request_id': 'req-39cdb946-a595-4d4c-8f34-2df0a0cc0ee6'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:08.489815 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-39cdb946-a595-4d4c-8f34-2df0a0cc0ee6 admin admin] The token cache is being invalidate because project fe13e91845924ce3bd4c49e6ff7f0db1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 13:47:08.490196 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-39cdb946-a595-4d4c-8f34-2df0a0cc0ee6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:08.490904 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-39cdb946-a595-4d4c-8f34-2df0a0cc0ee6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:08.492490 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 988/1983] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 13:47:08 2022] DELETE /identity/v3/projects/fe13e91845924ce3bd4c49e6ff7f0db1 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:47:08.496855 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74630c4b-dd43-4e23-8f1e-130dc5222a0e service neutron] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:08.507246 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74630c4b-dd43-4e23-8f1e-130dc5222a0e service neutron] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:08.571642 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8c597b-286f-4402-bdc9-6945c9b52ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:08.572355 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8c597b-286f-4402-bdc9-6945c9b52ec3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:08.572642 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8c597b-286f-4402-bdc9-6945c9b52ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:08.572922 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8c597b-286f-4402-bdc9-6945c9b52ec3 admin admin] PATH_INFO: `/v3/users/a1d2e257aa8543239d8f2e4778b4c6d8` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:08.574117 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8c597b-286f-4402-bdc9-6945c9b52ec3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1d2e257aa8543239d8f2e4778b4c6d8)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:08.576070 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8c597b-286f-4402-bdc9-6945c9b52ec3 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:08.623505 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-1d8c597b-286f-4402-bdc9-6945c9b52ec3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1d2e257aa8543239d8f2e4778b4c6d8'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:08.630272 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-1d8c597b-286f-4402-bdc9-6945c9b52ec3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1d2e257aa8543239d8f2e4778b4c6d8'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:08.637141 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-1d8c597b-286f-4402-bdc9-6945c9b52ec3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1d2e257aa8543239d8f2e4778b4c6d8'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:08.637786 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-1d8c597b-286f-4402-bdc9-6945c9b52ec3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1d2e257aa8543239d8f2e4778b4c6d8'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:08.638393 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-1d8c597b-286f-4402-bdc9-6945c9b52ec3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1d2e257aa8543239d8f2e4778b4c6d8'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:08.657826 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 996/1984] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 13:47:08 2022] DELETE /identity/v3/users/a1d2e257aa8543239d8f2e4778b4c6d8 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:47:08.943745 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cdb946-a595-4d4c-8f34-2df0a0cc0ee6 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:08.945219 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-39cdb946-a595-4d4c-8f34-2df0a0cc0ee6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:08.959579 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cdb946-a595-4d4c-8f34-2df0a0cc0ee6 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:09.018580 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b10f7e-ec76-4f8c-9db5-77f6b90b88a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:09.019160 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b10f7e-ec76-4f8c-9db5-77f6b90b88a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:09.019441 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b10f7e-ec76-4f8c-9db5-77f6b90b88a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:09.019721 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b10f7e-ec76-4f8c-9db5-77f6b90b88a0 admin admin] PATH_INFO: `/v3/projects/44f686a9aad14f99974057b16d585f8d` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:09.020869 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b10f7e-ec76-4f8c-9db5-77f6b90b88a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44f686a9aad14f99974057b16d585f8d)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:09.022797 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b10f7e-ec76-4f8c-9db5-77f6b90b88a0 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:09.074551 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-b8b10f7e-ec76-4f8c-9db5-77f6b90b88a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44f686a9aad14f99974057b16d585f8d', 'request_id': 'req-b8b10f7e-ec76-4f8c-9db5-77f6b90b88a0'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:09.081129 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-b8b10f7e-ec76-4f8c-9db5-77f6b90b88a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44f686a9aad14f99974057b16d585f8d', 'request_id': 'req-b8b10f7e-ec76-4f8c-9db5-77f6b90b88a0'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:09.087278 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-b8b10f7e-ec76-4f8c-9db5-77f6b90b88a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44f686a9aad14f99974057b16d585f8d', 'request_id': 'req-b8b10f7e-ec76-4f8c-9db5-77f6b90b88a0'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:09.100220 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-b8b10f7e-ec76-4f8c-9db5-77f6b90b88a0 admin admin] The token cache is being invalidate because project 44f686a9aad14f99974057b16d585f8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 13:47:09.100495 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-b8b10f7e-ec76-4f8c-9db5-77f6b90b88a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:09.101100 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-b8b10f7e-ec76-4f8c-9db5-77f6b90b88a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:09.102999 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 989/1985] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 13:47:08 2022] DELETE /identity/v3/projects/44f686a9aad14f99974057b16d585f8d => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:47:09.107874 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5718b097-6953-458e-9418-572afd2f9386 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:09.108336 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5718b097-6953-458e-9418-572afd2f9386 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:09.108454 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5718b097-6953-458e-9418-572afd2f9386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:09.129012 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-5718b097-6953-458e-9418-572afd2f9386 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:47:09.130259 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-5718b097-6953-458e-9418-572afd2f9386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:09.180960 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:47:09.192137 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 997/1986] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:47:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:47:09.199836 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493e0b1d-25dc-4293-a3cb-f886eb9a726d None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:09.200127 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493e0b1d-25dc-4293-a3cb-f886eb9a726d None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:09.200406 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493e0b1d-25dc-4293-a3cb-f886eb9a726d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:09.215133 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:47:09.222113 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-493e0b1d-25dc-4293-a3cb-f886eb9a726d None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:47:09.223962 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-493e0b1d-25dc-4293-a3cb-f886eb9a726d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:09.231839 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 990/1987] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:47:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:47:09.236426 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5718b097-6953-458e-9418-572afd2f9386 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:09.245834 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5718b097-6953-458e-9418-572afd2f9386 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:09.251203 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559d6460-97a7-43d6-af72-fc6e0c4cc1e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:09.251809 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559d6460-97a7-43d6-af72-fc6e0c4cc1e1 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:09.252088 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559d6460-97a7-43d6-af72-fc6e0c4cc1e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:09.252358 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559d6460-97a7-43d6-af72-fc6e0c4cc1e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:09.253042 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559d6460-97a7-43d6-af72-fc6e0c4cc1e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:47:09.253345 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559d6460-97a7-43d6-af72-fc6e0c4cc1e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:09.254962 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559d6460-97a7-43d6-af72-fc6e0c4cc1e1 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:09.266462 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 998/1988] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Jul 26 13:47:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:47:09.270614 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493e0b1d-25dc-4293-a3cb-f886eb9a726d None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:09.279590 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493e0b1d-25dc-4293-a3cb-f886eb9a726d None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:09.284933 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d61b16-dc58-4bff-a073-23587465757b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:09.285566 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d61b16-dc58-4bff-a073-23587465757b admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:09.285847 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d61b16-dc58-4bff-a073-23587465757b admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:09.286122 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d61b16-dc58-4bff-a073-23587465757b admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:09.286837 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d61b16-dc58-4bff-a073-23587465757b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:09.288689 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d61b16-dc58-4bff-a073-23587465757b admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:09.320669 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 991/1989] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 13:47:09 2022] POST /identity/v3/projects => generated 389 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:47:09.321871 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559d6460-97a7-43d6-af72-fc6e0c4cc1e1 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:09.331885 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559d6460-97a7-43d6-af72-fc6e0c4cc1e1 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:09.392249 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc2a971-bef9-4546-9ea6-51ce54e3b438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:09.392249 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc2a971-bef9-4546-9ea6-51ce54e3b438 admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:09.392249 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc2a971-bef9-4546-9ea6-51ce54e3b438 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:09.392249 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc2a971-bef9-4546-9ea6-51ce54e3b438 admin admin] PATH_INFO: `/v3/users` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:09.392576 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc2a971-bef9-4546-9ea6-51ce54e3b438 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:09.392576 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc2a971-bef9-4546-9ea6-51ce54e3b438 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:09.412844 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 999/1990] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 13:47:09 2022] POST /identity/v3/users => generated 295 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:47:09.416977 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d61b16-dc58-4bff-a073-23587465757b admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:09.427902 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d61b16-dc58-4bff-a073-23587465757b admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:09.432533 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ca1625-0b88-4984-a86e-2dd34e30d58a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:09.433146 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ca1625-0b88-4984-a86e-2dd34e30d58a admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:09.433428 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ca1625-0b88-4984-a86e-2dd34e30d58a admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:09.433735 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ca1625-0b88-4984-a86e-2dd34e30d58a admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:09.434429 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ca1625-0b88-4984-a86e-2dd34e30d58a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:09.436038 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ca1625-0b88-4984-a86e-2dd34e30d58a admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:09.445876 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 992/1991] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:47:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:47:09.449621 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc2a971-bef9-4546-9ea6-51ce54e3b438 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:09.459213 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc2a971-bef9-4546-9ea6-51ce54e3b438 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:09.463682 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687c04d9-4fdb-4b9b-b9f4-fda4b2ff0e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:09.464282 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687c04d9-4fdb-4b9b-b9f4-fda4b2ff0e8a admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:09.464561 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687c04d9-4fdb-4b9b-b9f4-fda4b2ff0e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:09.464834 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687c04d9-4fdb-4b9b-b9f4-fda4b2ff0e8a admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:09.465556 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687c04d9-4fdb-4b9b-b9f4-fda4b2ff0e8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:09.467151 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687c04d9-4fdb-4b9b-b9f4-fda4b2ff0e8a admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:09.478095 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1000/1992] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:47:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:47:09.481980 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ca1625-0b88-4984-a86e-2dd34e30d58a admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:09.492435 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ca1625-0b88-4984-a86e-2dd34e30d58a admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:09.497060 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa74d83f-a20f-493a-ab83-a1193e5e436e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:09.497756 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa74d83f-a20f-493a-ab83-a1193e5e436e admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:09.498041 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa74d83f-a20f-493a-ab83-a1193e5e436e admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:09.498319 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa74d83f-a20f-493a-ab83-a1193e5e436e admin admin] PATH_INFO: `/v3/projects/0f46e4161fd14df8a2c86857b20a6d2d/users/9b6603042a734fe6a5ac7b62eb959090/roles/759f389b221049c4b01faf3b984cd8d7` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:09.513607 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa74d83f-a20f-493a-ab83-a1193e5e436e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f46e4161fd14df8a2c86857b20a6d2d, user_id=9b6603042a734fe6a5ac7b62eb959090, role_id=759f389b221049c4b01faf3b984cd8d7)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:09.516252 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa74d83f-a20f-493a-ab83-a1193e5e436e admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:09.528558 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 993/1993] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 13:47:09 2022] PUT /identity/v3/projects/0f46e4161fd14df8a2c86857b20a6d2d/users/9b6603042a734fe6a5ac7b62eb959090/roles/759f389b221049c4b01faf3b984cd8d7 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:47:09.540184 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae91cca-e9ce-42d4-b1ed-31ab2b6350d8 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:09.540457 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae91cca-e9ce-42d4-b1ed-31ab2b6350d8 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:09.540730 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae91cca-e9ce-42d4-b1ed-31ab2b6350d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:09.570430 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-5ae91cca-e9ce-42d4-b1ed-31ab2b6350d8 None None] MFA Rules not processed for user `9b6603042a734fe6a5ac7b62eb959090`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:47:09.571700 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-5ae91cca-e9ce-42d4-b1ed-31ab2b6350d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:09.641396 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1001/1994] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:47:09 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:47:09.652171 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa74d83f-a20f-493a-ab83-a1193e5e436e admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:09.653906 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-fa74d83f-a20f-493a-ab83-a1193e5e436e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:09.712538 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa74d83f-a20f-493a-ab83-a1193e5e436e admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:09.716168 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d984bf3a-9706-4c86-a094-2ba6cdeafdc6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:09.716793 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d984bf3a-9706-4c86-a094-2ba6cdeafdc6 service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:09.717076 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d984bf3a-9706-4c86-a094-2ba6cdeafdc6 service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:09.717494 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d984bf3a-9706-4c86-a094-2ba6cdeafdc6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:09.726126 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d984bf3a-9706-4c86-a094-2ba6cdeafdc6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:09.727664 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d984bf3a-9706-4c86-a094-2ba6cdeafdc6 service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:09.737259 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 994/1995] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:47:09 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:47:19.517239 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae91cca-e9ce-42d4-b1ed-31ab2b6350d8 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:19.518695 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-5ae91cca-e9ce-42d4-b1ed-31ab2b6350d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:19.531127 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae91cca-e9ce-42d4-b1ed-31ab2b6350d8 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:19.534689 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20510a9c-9e07-4cd7-a040-12ac662d8f81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:19.535304 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20510a9c-9e07-4cd7-a040-12ac662d8f81 service cinder] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:19.535590 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20510a9c-9e07-4cd7-a040-12ac662d8f81 service cinder] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:19.535871 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20510a9c-9e07-4cd7-a040-12ac662d8f81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:19.537643 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-20510a9c-9e07-4cd7-a040-12ac662d8f81 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:19.592014 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:47:19.595517 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20510a9c-9e07-4cd7-a040-12ac662d8f81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:19.598068 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20510a9c-9e07-4cd7-a040-12ac662d8f81 service cinder] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:19.607203 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1002/1996] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:47:19 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:47:19.732591 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=50,keystone:INSERT=2 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:47:19.748465 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d984bf3a-9706-4c86-a094-2ba6cdeafdc6 service cinder] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:19.749976 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-d984bf3a-9706-4c86-a094-2ba6cdeafdc6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:19.835516 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d984bf3a-9706-4c86-a094-2ba6cdeafdc6 service cinder] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:19.840225 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3b8b75-2e94-4c31-bf46-5b944124b823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:19.840901 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3b8b75-2e94-4c31-bf46-5b944124b823 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:19.841372 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3b8b75-2e94-4c31-bf46-5b944124b823 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:19.841613 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3b8b75-2e94-4c31-bf46-5b944124b823 admin admin] PATH_INFO: `/v3/users/00cbd308d03d480f87fb6c23273dbe22` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:19.842891 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3b8b75-2e94-4c31-bf46-5b944124b823 admin admin] RBAC: Authorizing `identity:delete_user(user_id=00cbd308d03d480f87fb6c23273dbe22)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:19.844760 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3b8b75-2e94-4c31-bf46-5b944124b823 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:19.905448 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-5b3b8b75-2e94-4c31-bf46-5b944124b823 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00cbd308d03d480f87fb6c23273dbe22'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:19.906542 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-5b3b8b75-2e94-4c31-bf46-5b944124b823 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00cbd308d03d480f87fb6c23273dbe22'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:19.916113 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-5b3b8b75-2e94-4c31-bf46-5b944124b823 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00cbd308d03d480f87fb6c23273dbe22'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:19.916113 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-5b3b8b75-2e94-4c31-bf46-5b944124b823 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00cbd308d03d480f87fb6c23273dbe22'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:19.933787 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-5b3b8b75-2e94-4c31-bf46-5b944124b823 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00cbd308d03d480f87fb6c23273dbe22'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:19.951538 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 995/1997] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 13:47:19 2022] DELETE /identity/v3/users/00cbd308d03d480f87fb6c23273dbe22 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:47:19.956669 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a39b9d-266a-408c-96ad-d8c48865ca1e None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:19.957100 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a39b9d-266a-408c-96ad-d8c48865ca1e None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:19.957617 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a39b9d-266a-408c-96ad-d8c48865ca1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:19.976303 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-30a39b9d-266a-408c-96ad-d8c48865ca1e None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:47:19.977637 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-30a39b9d-266a-408c-96ad-d8c48865ca1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:20.045168 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1003/1998] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:47:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:47:20.051920 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3b8b75-2e94-4c31-bf46-5b944124b823 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:20.053320 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-5b3b8b75-2e94-4c31-bf46-5b944124b823 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:20.111648 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3b8b75-2e94-4c31-bf46-5b944124b823 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:20.115302 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39425567-73f0-475d-a61a-c6896a757e1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:20.126155 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39425567-73f0-475d-a61a-c6896a757e1d service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:20.126565 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39425567-73f0-475d-a61a-c6896a757e1d service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:20.126939 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39425567-73f0-475d-a61a-c6896a757e1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:20.138502 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39425567-73f0-475d-a61a-c6896a757e1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:20.140085 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39425567-73f0-475d-a61a-c6896a757e1d service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:20.152124 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 996/1999] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:47:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:47:20.466347 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a39b9d-266a-408c-96ad-d8c48865ca1e None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:20.468020 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-30a39b9d-266a-408c-96ad-d8c48865ca1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:20.481800 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a39b9d-266a-408c-96ad-d8c48865ca1e None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:20.486901 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849517d6-9210-45db-aa0d-fc95116b2d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:20.487551 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849517d6-9210-45db-aa0d-fc95116b2d7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:20.487830 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849517d6-9210-45db-aa0d-fc95116b2d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:20.488154 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849517d6-9210-45db-aa0d-fc95116b2d7e admin admin] PATH_INFO: `/v3/projects/52ce91fedeb5457c996654431ae78c69` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:20.489275 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849517d6-9210-45db-aa0d-fc95116b2d7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=52ce91fedeb5457c996654431ae78c69)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:20.491198 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849517d6-9210-45db-aa0d-fc95116b2d7e admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:20.537487 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-849517d6-9210-45db-aa0d-fc95116b2d7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52ce91fedeb5457c996654431ae78c69', 'request_id': 'req-849517d6-9210-45db-aa0d-fc95116b2d7e'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:20.544776 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-849517d6-9210-45db-aa0d-fc95116b2d7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52ce91fedeb5457c996654431ae78c69', 'request_id': 'req-849517d6-9210-45db-aa0d-fc95116b2d7e'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:20.550464 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-849517d6-9210-45db-aa0d-fc95116b2d7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52ce91fedeb5457c996654431ae78c69', 'request_id': 'req-849517d6-9210-45db-aa0d-fc95116b2d7e'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:20.561951 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-849517d6-9210-45db-aa0d-fc95116b2d7e admin admin] The token cache is being invalidate because project 52ce91fedeb5457c996654431ae78c69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86933) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 13:47:20.562444 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-849517d6-9210-45db-aa0d-fc95116b2d7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:20.563269 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-849517d6-9210-45db-aa0d-fc95116b2d7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:20.568254 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1004/2000] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 13:47:20 2022] DELETE /identity/v3/projects/52ce91fedeb5457c996654431ae78c69 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:47:20.569928 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6959b7-7070-4718-bb33-9463601e4120 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:20.570228 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6959b7-7070-4718-bb33-9463601e4120 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:20.570517 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6959b7-7070-4718-bb33-9463601e4120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:20.590311 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-8b6959b7-7070-4718-bb33-9463601e4120 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:47:20.591515 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-8b6959b7-7070-4718-bb33-9463601e4120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:20.608153 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:47:20.653054 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 997/2001] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:47:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 13:47:20.661648 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471dac87-41c3-4582-a09b-8ae4851b0b89 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:20.661940 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471dac87-41c3-4582-a09b-8ae4851b0b89 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:20.662224 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471dac87-41c3-4582-a09b-8ae4851b0b89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:20.682232 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-471dac87-41c3-4582-a09b-8ae4851b0b89 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:47:20.683417 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-471dac87-41c3-4582-a09b-8ae4851b0b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:20.691045 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1005/2002] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:47:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:47:20.695192 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6959b7-7070-4718-bb33-9463601e4120 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:20.704989 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6959b7-7070-4718-bb33-9463601e4120 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:20.709500 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cbcbdd-087c-4db3-9e54-01a1a455c012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:20.710120 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cbcbdd-087c-4db3-9e54-01a1a455c012 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:20.710406 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cbcbdd-087c-4db3-9e54-01a1a455c012 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:20.710707 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cbcbdd-087c-4db3-9e54-01a1a455c012 admin admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:20.711406 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cbcbdd-087c-4db3-9e54-01a1a455c012 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:47:20.711711 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cbcbdd-087c-4db3-9e54-01a1a455c012 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:20.713296 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cbcbdd-087c-4db3-9e54-01a1a455c012 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:20.725372 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 998/2003] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Jul 26 13:47:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:47:20.729355 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471dac87-41c3-4582-a09b-8ae4851b0b89 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:20.738880 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471dac87-41c3-4582-a09b-8ae4851b0b89 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:20.743370 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829b78fc-ddff-4e9a-ab3a-f088d78c5960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:20.744256 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829b78fc-ddff-4e9a-ab3a-f088d78c5960 admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:20.744547 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829b78fc-ddff-4e9a-ab3a-f088d78c5960 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:20.744819 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829b78fc-ddff-4e9a-ab3a-f088d78c5960 admin admin] PATH_INFO: `/v3/projects` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:20.745501 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829b78fc-ddff-4e9a-ab3a-f088d78c5960 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:20.747370 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829b78fc-ddff-4e9a-ab3a-f088d78c5960 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:20.774364 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1006/2004] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 13:47:20 2022] POST /identity/v3/projects => generated 375 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 13:47:20.778411 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cbcbdd-087c-4db3-9e54-01a1a455c012 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:20.788915 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cbcbdd-087c-4db3-9e54-01a1a455c012 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:20.847230 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6f5750-f017-4f3a-86cc-67b4df99f6c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:20.847850 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6f5750-f017-4f3a-86cc-67b4df99f6c5 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:20.848134 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6f5750-f017-4f3a-86cc-67b4df99f6c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:20.848410 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6f5750-f017-4f3a-86cc-67b4df99f6c5 admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:20.849079 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6f5750-f017-4f3a-86cc-67b4df99f6c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:20.851051 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6f5750-f017-4f3a-86cc-67b4df99f6c5 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:20.873093 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 999/2005] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 13:47:20 2022] POST /identity/v3/users => generated 288 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:47:20.877170 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829b78fc-ddff-4e9a-ab3a-f088d78c5960 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:20.887291 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829b78fc-ddff-4e9a-ab3a-f088d78c5960 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:20.892658 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784fc0ed-7e17-4d79-8622-f78c881e2429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:20.893157 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784fc0ed-7e17-4d79-8622-f78c881e2429 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:20.893436 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784fc0ed-7e17-4d79-8622-f78c881e2429 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:20.893739 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784fc0ed-7e17-4d79-8622-f78c881e2429 admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:20.894434 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784fc0ed-7e17-4d79-8622-f78c881e2429 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:20.896029 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784fc0ed-7e17-4d79-8622-f78c881e2429 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:20.906856 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1007/2006] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:47:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:47:20.910779 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6f5750-f017-4f3a-86cc-67b4df99f6c5 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:20.920885 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6f5750-f017-4f3a-86cc-67b4df99f6c5 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:20.925420 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2a1b69-fd7b-424a-9937-c61bf363ffda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:20.926076 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2a1b69-fd7b-424a-9937-c61bf363ffda admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:20.926356 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2a1b69-fd7b-424a-9937-c61bf363ffda admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:20.926668 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2a1b69-fd7b-424a-9937-c61bf363ffda admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:20.927704 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2a1b69-fd7b-424a-9937-c61bf363ffda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:20.929341 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2a1b69-fd7b-424a-9937-c61bf363ffda admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:20.940804 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1000/2007] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:47:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 13:47:20.944849 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784fc0ed-7e17-4d79-8622-f78c881e2429 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:20.955713 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784fc0ed-7e17-4d79-8622-f78c881e2429 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:20.960350 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347c3f37-b542-44b3-8f3d-6b4f7afae688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:20.961077 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347c3f37-b542-44b3-8f3d-6b4f7afae688 admin admin] REQUEST_METHOD: `PUT` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:20.961348 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347c3f37-b542-44b3-8f3d-6b4f7afae688 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:20.961661 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347c3f37-b542-44b3-8f3d-6b4f7afae688 admin admin] PATH_INFO: `/v3/projects/615000c9b213489bb17bc95eb475006f/users/ceda9da793df4350a9c002ca2c785843/roles/759f389b221049c4b01faf3b984cd8d7` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:20.978079 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347c3f37-b542-44b3-8f3d-6b4f7afae688 admin admin] RBAC: Authorizing `identity:create_grant(project_id=615000c9b213489bb17bc95eb475006f, user_id=ceda9da793df4350a9c002ca2c785843, role_id=759f389b221049c4b01faf3b984cd8d7)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:20.980720 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347c3f37-b542-44b3-8f3d-6b4f7afae688 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:20.992875 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1008/2008] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 13:47:20 2022] PUT /identity/v3/projects/615000c9b213489bb17bc95eb475006f/users/ceda9da793df4350a9c002ca2c785843/roles/759f389b221049c4b01faf3b984cd8d7 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:47:21.003291 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58df706-1697-4b41-8c12-2c4b02cd8d76 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:21.003580 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58df706-1697-4b41-8c12-2c4b02cd8d76 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:21.003855 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58df706-1697-4b41-8c12-2c4b02cd8d76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:21.033540 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-e58df706-1697-4b41-8c12-2c4b02cd8d76 None None] MFA Rules not processed for user `ceda9da793df4350a9c002ca2c785843`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:47:21.034846 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-e58df706-1697-4b41-8c12-2c4b02cd8d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:21.113846 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1001/2009] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:47:21 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:47:21.118701 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347c3f37-b542-44b3-8f3d-6b4f7afae688 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:21.131817 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:47:21.183520 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347c3f37-b542-44b3-8f3d-6b4f7afae688 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:21.188550 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92e1545-1c55-4089-9f78-deefdea30559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:21.189245 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92e1545-1c55-4089-9f78-deefdea30559 admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:21.189559 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92e1545-1c55-4089-9f78-deefdea30559 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:21.189832 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92e1545-1c55-4089-9f78-deefdea30559 admin admin] PATH_INFO: `/v3/projects` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:21.190509 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92e1545-1c55-4089-9f78-deefdea30559 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:21.192396 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92e1545-1c55-4089-9f78-deefdea30559 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:21.224546 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1009/2010] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 13:47:21 2022] POST /identity/v3/projects => generated 373 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:47:21.225849 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58df706-1697-4b41-8c12-2c4b02cd8d76 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:21.237147 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58df706-1697-4b41-8c12-2c4b02cd8d76 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:21.254691 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:47:21.312634 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01233c08-bdbe-47c2-b07a-8e608fcc5105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:21.313272 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01233c08-bdbe-47c2-b07a-8e608fcc5105 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:21.313593 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01233c08-bdbe-47c2-b07a-8e608fcc5105 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:21.313873 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01233c08-bdbe-47c2-b07a-8e608fcc5105 admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:21.314552 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01233c08-bdbe-47c2-b07a-8e608fcc5105 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:21.316510 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01233c08-bdbe-47c2-b07a-8e608fcc5105 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:21.341596 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1002/2011] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 13:47:21 2022] POST /identity/v3/users => generated 293 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:47:21.342918 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92e1545-1c55-4089-9f78-deefdea30559 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:21.353024 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92e1545-1c55-4089-9f78-deefdea30559 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:21.357707 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cb7512-0ec3-415a-be39-dd2fcae80a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:21.358410 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb7512-0ec3-415a-be39-dd2fcae80a3f admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:21.358688 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb7512-0ec3-415a-be39-dd2fcae80a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:21.359021 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb7512-0ec3-415a-be39-dd2fcae80a3f admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:21.359727 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cb7512-0ec3-415a-be39-dd2fcae80a3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:21.361296 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cb7512-0ec3-415a-be39-dd2fcae80a3f admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:21.371968 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1010/2012] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:47:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:47:21.376751 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01233c08-bdbe-47c2-b07a-8e608fcc5105 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:21.386691 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01233c08-bdbe-47c2-b07a-8e608fcc5105 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:21.392174 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915b958f-50a6-4f3d-93bf-fbfdeea1363c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:21.392880 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915b958f-50a6-4f3d-93bf-fbfdeea1363c admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:21.393162 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915b958f-50a6-4f3d-93bf-fbfdeea1363c admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:21.393512 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915b958f-50a6-4f3d-93bf-fbfdeea1363c admin admin] PATH_INFO: `/v3/projects/c2cca54a1aeb45c4ab505f2ecea78549/users/9da95b5111c4414e9b1830351f498a30/roles/6e0201a9a6484f7d8b112b1eafff70bd` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:21.411157 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915b958f-50a6-4f3d-93bf-fbfdeea1363c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2cca54a1aeb45c4ab505f2ecea78549, user_id=9da95b5111c4414e9b1830351f498a30, role_id=6e0201a9a6484f7d8b112b1eafff70bd)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:21.413840 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915b958f-50a6-4f3d-93bf-fbfdeea1363c admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:21.432524 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1003/2013] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 13:47:21 2022] PUT /identity/v3/projects/c2cca54a1aeb45c4ab505f2ecea78549/users/9da95b5111c4414e9b1830351f498a30/roles/6e0201a9a6484f7d8b112b1eafff70bd => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:47:21.438232 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cb7512-0ec3-415a-be39-dd2fcae80a3f admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:21.448063 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cb7512-0ec3-415a-be39-dd2fcae80a3f admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:21.516364 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba512528-f496-4b89-ab94-60a3485aef28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:21.516991 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba512528-f496-4b89-ab94-60a3485aef28 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:21.517273 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba512528-f496-4b89-ab94-60a3485aef28 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:21.517583 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba512528-f496-4b89-ab94-60a3485aef28 admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:21.518274 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba512528-f496-4b89-ab94-60a3485aef28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:21.519895 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba512528-f496-4b89-ab94-60a3485aef28 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:21.529366 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1011/2014] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:47:21 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:47:21.534287 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915b958f-50a6-4f3d-93bf-fbfdeea1363c admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:21.544274 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915b958f-50a6-4f3d-93bf-fbfdeea1363c admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:21.549078 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4cebd4-1c6b-4bb2-8d5e-5ed469416918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:21.549776 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4cebd4-1c6b-4bb2-8d5e-5ed469416918 admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:21.550063 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4cebd4-1c6b-4bb2-8d5e-5ed469416918 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:21.550473 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4cebd4-1c6b-4bb2-8d5e-5ed469416918 admin admin] PATH_INFO: `/v3/projects/c2cca54a1aeb45c4ab505f2ecea78549/users/9da95b5111c4414e9b1830351f498a30/roles/6e0201a9a6484f7d8b112b1eafff70bd` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:21.552410 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4cebd4-1c6b-4bb2-8d5e-5ed469416918 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2cca54a1aeb45c4ab505f2ecea78549, user_id=9da95b5111c4414e9b1830351f498a30, role_id=6e0201a9a6484f7d8b112b1eafff70bd)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:21.555056 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4cebd4-1c6b-4bb2-8d5e-5ed469416918 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:21.565959 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1004/2015] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 13:47:21 2022] PUT /identity/v3/projects/c2cca54a1aeb45c4ab505f2ecea78549/users/9da95b5111c4414e9b1830351f498a30/roles/6e0201a9a6484f7d8b112b1eafff70bd => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:47:21.576835 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62edfdbe-dda4-42a3-af3c-6ba5f597273f None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:21.577950 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62edfdbe-dda4-42a3-af3c-6ba5f597273f None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:21.578113 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62edfdbe-dda4-42a3-af3c-6ba5f597273f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:21.604691 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-62edfdbe-dda4-42a3-af3c-6ba5f597273f None None] MFA Rules not processed for user `9da95b5111c4414e9b1830351f498a30`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:47:21.605934 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-62edfdbe-dda4-42a3-af3c-6ba5f597273f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:21.673346 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1012/2016] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:47:21 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:47:21.684129 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4cebd4-1c6b-4bb2-8d5e-5ed469416918 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:21.685848 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-ed4cebd4-1c6b-4bb2-8d5e-5ed469416918 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:21.741890 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4cebd4-1c6b-4bb2-8d5e-5ed469416918 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:21.745506 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd41d5c-d66a-4bec-b986-facc4e1d4c41 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:21.746113 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd41d5c-d66a-4bec-b986-facc4e1d4c41 service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:21.746392 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd41d5c-d66a-4bec-b986-facc4e1d4c41 service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:21.746661 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd41d5c-d66a-4bec-b986-facc4e1d4c41 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:21.805772 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd41d5c-d66a-4bec-b986-facc4e1d4c41 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:21.807195 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd41d5c-d66a-4bec-b986-facc4e1d4c41 service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:21.816384 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1005/2017] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:47:21 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:47:23.826013 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62edfdbe-dda4-42a3-af3c-6ba5f597273f None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:23.827483 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-62edfdbe-dda4-42a3-af3c-6ba5f597273f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:23.841149 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:47:23.910199 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62edfdbe-dda4-42a3-af3c-6ba5f597273f None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:23.914692 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56db9590-d0f2-4a82-9db8-4de4d4d7e234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:23.916002 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56db9590-d0f2-4a82-9db8-4de4d4d7e234 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:23.916286 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56db9590-d0f2-4a82-9db8-4de4d4d7e234 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:23.916568 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56db9590-d0f2-4a82-9db8-4de4d4d7e234 admin admin] PATH_INFO: `/v3/users/f0284f920ee849e4a95b32626395096c` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:23.917757 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56db9590-d0f2-4a82-9db8-4de4d4d7e234 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0284f920ee849e4a95b32626395096c)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:23.919668 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56db9590-d0f2-4a82-9db8-4de4d4d7e234 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:23.978269 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-56db9590-d0f2-4a82-9db8-4de4d4d7e234 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0284f920ee849e4a95b32626395096c'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:23.987487 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-56db9590-d0f2-4a82-9db8-4de4d4d7e234 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0284f920ee849e4a95b32626395096c'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:23.995410 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-56db9590-d0f2-4a82-9db8-4de4d4d7e234 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0284f920ee849e4a95b32626395096c'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:23.996096 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-56db9590-d0f2-4a82-9db8-4de4d4d7e234 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0284f920ee849e4a95b32626395096c'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:23.996724 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-56db9590-d0f2-4a82-9db8-4de4d4d7e234 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0284f920ee849e4a95b32626395096c'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:24.020289 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1013/2018] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 13:47:23 2022] DELETE /identity/v3/users/f0284f920ee849e4a95b32626395096c => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:47:24.024519 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f3067c-1093-4487-951f-fe9558bd2c88 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:24.025379 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f3067c-1093-4487-951f-fe9558bd2c88 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:24.025833 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f3067c-1093-4487-951f-fe9558bd2c88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:24.059434 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-f2f3067c-1093-4487-951f-fe9558bd2c88 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:47:24.060889 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-f2f3067c-1093-4487-951f-fe9558bd2c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:24.085595 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:47:24.144554 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1006/2019] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:47:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 13:47:24.157844 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56db9590-d0f2-4a82-9db8-4de4d4d7e234 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:24.161657 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-56db9590-d0f2-4a82-9db8-4de4d4d7e234 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:24.269420 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56db9590-d0f2-4a82-9db8-4de4d4d7e234 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:24.273202 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e09fc68-ff53-4f29-afa7-adc7956d5fdf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:24.274015 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e09fc68-ff53-4f29-afa7-adc7956d5fdf service neutron] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:24.274305 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e09fc68-ff53-4f29-afa7-adc7956d5fdf service neutron] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:24.274617 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e09fc68-ff53-4f29-afa7-adc7956d5fdf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:24.291457 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e09fc68-ff53-4f29-afa7-adc7956d5fdf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:24.293348 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e09fc68-ff53-4f29-afa7-adc7956d5fdf service neutron] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:24.305330 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1014/2020] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:47:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:47:24.655787 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f3067c-1093-4487-951f-fe9558bd2c88 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:24.657255 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-f2f3067c-1093-4487-951f-fe9558bd2c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:24.670277 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f3067c-1093-4487-951f-fe9558bd2c88 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:24.675821 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d0a18f-a8c3-4ad3-82f6-c31f689a7acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:24.676581 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d0a18f-a8c3-4ad3-82f6-c31f689a7acf admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:24.676939 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d0a18f-a8c3-4ad3-82f6-c31f689a7acf admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:24.677292 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d0a18f-a8c3-4ad3-82f6-c31f689a7acf admin admin] PATH_INFO: `/v3/projects/15cb70e82f804eed9d479538f76ec027` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:24.678571 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d0a18f-a8c3-4ad3-82f6-c31f689a7acf admin admin] RBAC: Authorizing `identity:delete_project(project_id=15cb70e82f804eed9d479538f76ec027)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:24.680575 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d0a18f-a8c3-4ad3-82f6-c31f689a7acf admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:24.729830 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-f4d0a18f-a8c3-4ad3-82f6-c31f689a7acf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15cb70e82f804eed9d479538f76ec027', 'request_id': 'req-f4d0a18f-a8c3-4ad3-82f6-c31f689a7acf'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:24.737144 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-f4d0a18f-a8c3-4ad3-82f6-c31f689a7acf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15cb70e82f804eed9d479538f76ec027', 'request_id': 'req-f4d0a18f-a8c3-4ad3-82f6-c31f689a7acf'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:24.743047 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-f4d0a18f-a8c3-4ad3-82f6-c31f689a7acf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15cb70e82f804eed9d479538f76ec027', 'request_id': 'req-f4d0a18f-a8c3-4ad3-82f6-c31f689a7acf'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:24.756488 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-f4d0a18f-a8c3-4ad3-82f6-c31f689a7acf admin admin] The token cache is being invalidate because project 15cb70e82f804eed9d479538f76ec027 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 13:47:24.756786 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-f4d0a18f-a8c3-4ad3-82f6-c31f689a7acf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:24.757534 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-f4d0a18f-a8c3-4ad3-82f6-c31f689a7acf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:24.759203 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1007/2021] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 13:47:24 2022] DELETE /identity/v3/projects/15cb70e82f804eed9d479538f76ec027 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:47:24.763167 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e09fc68-ff53-4f29-afa7-adc7956d5fdf service neutron] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:24.773046 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e09fc68-ff53-4f29-afa7-adc7956d5fdf service neutron] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:24.803877 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:47:24.836013 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5732fadf-2057-4dda-8a45-62ca4d54916e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:24.836722 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5732fadf-2057-4dda-8a45-62ca4d54916e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:24.836983 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5732fadf-2057-4dda-8a45-62ca4d54916e admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:24.837271 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5732fadf-2057-4dda-8a45-62ca4d54916e admin admin] PATH_INFO: `/v3/users/377822fc1b6b41689428a60b16d2d9e4` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:24.838661 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5732fadf-2057-4dda-8a45-62ca4d54916e admin admin] RBAC: Authorizing `identity:delete_user(user_id=377822fc1b6b41689428a60b16d2d9e4)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:24.840591 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5732fadf-2057-4dda-8a45-62ca4d54916e admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:24.893074 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-5732fadf-2057-4dda-8a45-62ca4d54916e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '377822fc1b6b41689428a60b16d2d9e4'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:24.901867 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-5732fadf-2057-4dda-8a45-62ca4d54916e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '377822fc1b6b41689428a60b16d2d9e4'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:24.911528 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-5732fadf-2057-4dda-8a45-62ca4d54916e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '377822fc1b6b41689428a60b16d2d9e4'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:24.912121 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-5732fadf-2057-4dda-8a45-62ca4d54916e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '377822fc1b6b41689428a60b16d2d9e4'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:24.912851 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-5732fadf-2057-4dda-8a45-62ca4d54916e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '377822fc1b6b41689428a60b16d2d9e4'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:24.934897 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1015/2022] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 13:47:24 2022] DELETE /identity/v3/users/377822fc1b6b41689428a60b16d2d9e4 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:47:25.454435 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d0a18f-a8c3-4ad3-82f6-c31f689a7acf admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:25.456500 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-f4d0a18f-a8c3-4ad3-82f6-c31f689a7acf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:25.470987 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d0a18f-a8c3-4ad3-82f6-c31f689a7acf admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:25.540650 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66faabae-cc3e-4709-8798-906847b0e3a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:25.541343 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66faabae-cc3e-4709-8798-906847b0e3a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:25.541655 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66faabae-cc3e-4709-8798-906847b0e3a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:25.541930 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66faabae-cc3e-4709-8798-906847b0e3a2 admin admin] PATH_INFO: `/v3/projects/4ff1be3204ed49a09fa5012107116be0` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:25.543127 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66faabae-cc3e-4709-8798-906847b0e3a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ff1be3204ed49a09fa5012107116be0)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:25.545061 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66faabae-cc3e-4709-8798-906847b0e3a2 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:25.599493 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-66faabae-cc3e-4709-8798-906847b0e3a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ff1be3204ed49a09fa5012107116be0', 'request_id': 'req-66faabae-cc3e-4709-8798-906847b0e3a2'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:25.600573 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:47:25.608701 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-66faabae-cc3e-4709-8798-906847b0e3a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ff1be3204ed49a09fa5012107116be0', 'request_id': 'req-66faabae-cc3e-4709-8798-906847b0e3a2'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:25.615345 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-66faabae-cc3e-4709-8798-906847b0e3a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ff1be3204ed49a09fa5012107116be0', 'request_id': 'req-66faabae-cc3e-4709-8798-906847b0e3a2'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:25.628294 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-66faabae-cc3e-4709-8798-906847b0e3a2 admin admin] The token cache is being invalidate because project 4ff1be3204ed49a09fa5012107116be0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 13:47:25.628621 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-66faabae-cc3e-4709-8798-906847b0e3a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:25.629205 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-66faabae-cc3e-4709-8798-906847b0e3a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:25.630739 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1008/2023] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 13:47:25 2022] DELETE /identity/v3/projects/4ff1be3204ed49a09fa5012107116be0 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:47:25.634632 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5732fadf-2057-4dda-8a45-62ca4d54916e admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:25.636034 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-5732fadf-2057-4dda-8a45-62ca4d54916e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:25.650001 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5732fadf-2057-4dda-8a45-62ca4d54916e admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:25.706875 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b4da74-46db-468b-8725-dbf3d4d6a808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:25.707523 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b4da74-46db-468b-8725-dbf3d4d6a808 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:25.707802 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b4da74-46db-468b-8725-dbf3d4d6a808 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:25.708075 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b4da74-46db-468b-8725-dbf3d4d6a808 admin admin] PATH_INFO: `/v3/users/d5a4b512f7f44f22a7067bd8be76da35` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:25.709225 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b4da74-46db-468b-8725-dbf3d4d6a808 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5a4b512f7f44f22a7067bd8be76da35)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:25.711157 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b4da74-46db-468b-8725-dbf3d4d6a808 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:25.758242 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-d4b4da74-46db-468b-8725-dbf3d4d6a808 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5a4b512f7f44f22a7067bd8be76da35'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:25.761753 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:47:25.766774 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-d4b4da74-46db-468b-8725-dbf3d4d6a808 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5a4b512f7f44f22a7067bd8be76da35'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:25.775563 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-d4b4da74-46db-468b-8725-dbf3d4d6a808 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5a4b512f7f44f22a7067bd8be76da35'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:25.776219 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-d4b4da74-46db-468b-8725-dbf3d4d6a808 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5a4b512f7f44f22a7067bd8be76da35'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:25.776832 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-d4b4da74-46db-468b-8725-dbf3d4d6a808 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5a4b512f7f44f22a7067bd8be76da35'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:25.798616 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1016/2024] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 13:47:25 2022] DELETE /identity/v3/users/d5a4b512f7f44f22a7067bd8be76da35 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:47:26.078270 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66faabae-cc3e-4709-8798-906847b0e3a2 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:26.079738 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-66faabae-cc3e-4709-8798-906847b0e3a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:26.095156 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66faabae-cc3e-4709-8798-906847b0e3a2 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:26.164570 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40afa78-f142-4dee-aafb-0bdf54b6be7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:26.165345 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40afa78-f142-4dee-aafb-0bdf54b6be7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:26.165732 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40afa78-f142-4dee-aafb-0bdf54b6be7f admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:26.166072 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40afa78-f142-4dee-aafb-0bdf54b6be7f admin admin] PATH_INFO: `/v3/projects/5841eeababae4d9882f7a0680b875208` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:26.167356 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40afa78-f142-4dee-aafb-0bdf54b6be7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=5841eeababae4d9882f7a0680b875208)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:26.169358 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40afa78-f142-4dee-aafb-0bdf54b6be7f admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:26.229355 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-c40afa78-f142-4dee-aafb-0bdf54b6be7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5841eeababae4d9882f7a0680b875208', 'request_id': 'req-c40afa78-f142-4dee-aafb-0bdf54b6be7f'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:26.236769 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-c40afa78-f142-4dee-aafb-0bdf54b6be7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5841eeababae4d9882f7a0680b875208', 'request_id': 'req-c40afa78-f142-4dee-aafb-0bdf54b6be7f'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:26.247039 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-c40afa78-f142-4dee-aafb-0bdf54b6be7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5841eeababae4d9882f7a0680b875208', 'request_id': 'req-c40afa78-f142-4dee-aafb-0bdf54b6be7f'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:26.274661 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-c40afa78-f142-4dee-aafb-0bdf54b6be7f admin admin] The token cache is being invalidate because project 5841eeababae4d9882f7a0680b875208 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 13:47:26.275000 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-c40afa78-f142-4dee-aafb-0bdf54b6be7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:26.277920 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-c40afa78-f142-4dee-aafb-0bdf54b6be7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:26.291515 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41beb1d8-d4bf-4e75-8ba8-fb4db87ef741 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:26.292394 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41beb1d8-d4bf-4e75-8ba8-fb4db87ef741 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:26.292992 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41beb1d8-d4bf-4e75-8ba8-fb4db87ef741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:26.302740 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1009/2025] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 13:47:26 2022] DELETE /identity/v3/projects/5841eeababae4d9882f7a0680b875208 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:47:26.358550 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-41beb1d8-d4bf-4e75-8ba8-fb4db87ef741 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:47:26.362852 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-41beb1d8-d4bf-4e75-8ba8-fb4db87ef741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:26.469801 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1017/2026] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:47:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:47:26.477802 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1120dbb-fcd7-471b-b799-417075dc4332 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:26.478318 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1120dbb-fcd7-471b-b799-417075dc4332 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:26.478416 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1120dbb-fcd7-471b-b799-417075dc4332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:26.527331 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-a1120dbb-fcd7-471b-b799-417075dc4332 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:47:26.528869 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-a1120dbb-fcd7-471b-b799-417075dc4332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:26.544730 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41beb1d8-d4bf-4e75-8ba8-fb4db87ef741 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:26.563096 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41beb1d8-d4bf-4e75-8ba8-fb4db87ef741 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:26.563900 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1010/2027] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:47:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:47:26.576125 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e42ec13-d00b-4f06-991f-c9811987ffcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:26.577458 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e42ec13-d00b-4f06-991f-c9811987ffcd admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:26.577974 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e42ec13-d00b-4f06-991f-c9811987ffcd admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:26.578466 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e42ec13-d00b-4f06-991f-c9811987ffcd admin admin] PATH_INFO: `/v3/domains` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:26.579848 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e42ec13-d00b-4f06-991f-c9811987ffcd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:47:26.580347 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e42ec13-d00b-4f06-991f-c9811987ffcd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:26.582703 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e42ec13-d00b-4f06-991f-c9811987ffcd admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:26.612730 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1018/2028] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Jul 26 13:47:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:47:26.616715 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1120dbb-fcd7-471b-b799-417075dc4332 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:26.627898 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1120dbb-fcd7-471b-b799-417075dc4332 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:26.636374 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf629b4-9178-4d05-92aa-d2527ad9dcdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:26.637069 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf629b4-9178-4d05-92aa-d2527ad9dcdc admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:26.637396 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf629b4-9178-4d05-92aa-d2527ad9dcdc admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:26.637715 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf629b4-9178-4d05-92aa-d2527ad9dcdc admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:26.638659 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf629b4-9178-4d05-92aa-d2527ad9dcdc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:26.641647 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf629b4-9178-4d05-92aa-d2527ad9dcdc admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:26.689452 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1011/2029] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 13:47:26 2022] POST /identity/v3/projects => generated 373 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:47:26.690650 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e42ec13-d00b-4f06-991f-c9811987ffcd admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:26.701849 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e42ec13-d00b-4f06-991f-c9811987ffcd admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:26.760401 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcc122e-ed9f-4dcc-9416-0f7772f459a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:26.761026 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc122e-ed9f-4dcc-9416-0f7772f459a7 admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:26.761309 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc122e-ed9f-4dcc-9416-0f7772f459a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:26.761623 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc122e-ed9f-4dcc-9416-0f7772f459a7 admin admin] PATH_INFO: `/v3/users` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:26.762305 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcc122e-ed9f-4dcc-9416-0f7772f459a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:26.764243 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcc122e-ed9f-4dcc-9416-0f7772f459a7 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:26.787521 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1019/2030] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 13:47:26 2022] POST /identity/v3/users => generated 287 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:47:26.792293 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf629b4-9178-4d05-92aa-d2527ad9dcdc admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:26.805463 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf629b4-9178-4d05-92aa-d2527ad9dcdc admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:26.810627 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c70249-f44b-492f-bbd7-1d7dfedded13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:26.811285 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c70249-f44b-492f-bbd7-1d7dfedded13 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:26.811572 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c70249-f44b-492f-bbd7-1d7dfedded13 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:26.811846 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c70249-f44b-492f-bbd7-1d7dfedded13 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:26.812541 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c70249-f44b-492f-bbd7-1d7dfedded13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:26.814185 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c70249-f44b-492f-bbd7-1d7dfedded13 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:26.826612 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1012/2031] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:47:26 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:47:26.830446 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcc122e-ed9f-4dcc-9416-0f7772f459a7 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:26.845382 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcc122e-ed9f-4dcc-9416-0f7772f459a7 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:26.853233 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab06672d-1748-42f4-8825-aad33f370571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:26.854419 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab06672d-1748-42f4-8825-aad33f370571 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:26.854755 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab06672d-1748-42f4-8825-aad33f370571 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:26.855120 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab06672d-1748-42f4-8825-aad33f370571 admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:26.856266 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab06672d-1748-42f4-8825-aad33f370571 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:26.859389 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab06672d-1748-42f4-8825-aad33f370571 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:26.876466 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1020/2032] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:47:26 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:47:26.877329 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c70249-f44b-492f-bbd7-1d7dfedded13 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:26.890401 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c70249-f44b-492f-bbd7-1d7dfedded13 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:26.897244 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a13d82-cf50-4061-ae21-4ba920460bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:26.898162 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a13d82-cf50-4061-ae21-4ba920460bd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:26.898625 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a13d82-cf50-4061-ae21-4ba920460bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:26.898911 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a13d82-cf50-4061-ae21-4ba920460bd8 admin admin] PATH_INFO: `/v3/projects/6b3dd4e5489c47079f039821127998d6/users/0291c8ea030942558cb1012c0f427e6d/roles/759f389b221049c4b01faf3b984cd8d7` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:26.929764 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a13d82-cf50-4061-ae21-4ba920460bd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b3dd4e5489c47079f039821127998d6, user_id=0291c8ea030942558cb1012c0f427e6d, role_id=759f389b221049c4b01faf3b984cd8d7)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:26.933582 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a13d82-cf50-4061-ae21-4ba920460bd8 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:26.961335 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1013/2033] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 13:47:26 2022] PUT /identity/v3/projects/6b3dd4e5489c47079f039821127998d6/users/0291c8ea030942558cb1012c0f427e6d/roles/759f389b221049c4b01faf3b984cd8d7 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:47:26.975597 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1fa548-357f-4b5a-8ca8-b522927ded1b None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:26.975996 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1fa548-357f-4b5a-8ca8-b522927ded1b None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:26.976276 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1fa548-357f-4b5a-8ca8-b522927ded1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:27.019888 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-ac1fa548-357f-4b5a-8ca8-b522927ded1b None None] MFA Rules not processed for user `0291c8ea030942558cb1012c0f427e6d`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:47:27.021261 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-ac1fa548-357f-4b5a-8ca8-b522927ded1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:27.084400 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:47:27.132224 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1021/2034] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:47:26 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:47:27.143842 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a13d82-cf50-4061-ae21-4ba920460bd8 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:27.145579 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-16a13d82-cf50-4061-ae21-4ba920460bd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:27.152071 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:47:27.205741 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a13d82-cf50-4061-ae21-4ba920460bd8 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:27.209436 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b8b69e-7a24-4b0c-8e1e-7c7bae9067f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:27.210024 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b8b69e-7a24-4b0c-8e1e-7c7bae9067f1 service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:27.210305 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b8b69e-7a24-4b0c-8e1e-7c7bae9067f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:27.210580 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b8b69e-7a24-4b0c-8e1e-7c7bae9067f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:27.219742 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b8b69e-7a24-4b0c-8e1e-7c7bae9067f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:27.221165 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b8b69e-7a24-4b0c-8e1e-7c7bae9067f1 service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:27.229831 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1014/2035] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:47:27 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:47:27.649216 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1fa548-357f-4b5a-8ca8-b522927ded1b None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:27.651009 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-ac1fa548-357f-4b5a-8ca8-b522927ded1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:27.728019 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1fa548-357f-4b5a-8ca8-b522927ded1b None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:27.734387 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc92e13-f49b-4ee0-8291-e780cc03a1fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:27.735277 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc92e13-f49b-4ee0-8291-e780cc03a1fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:27.735891 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc92e13-f49b-4ee0-8291-e780cc03a1fa admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:27.736500 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc92e13-f49b-4ee0-8291-e780cc03a1fa admin admin] PATH_INFO: `/v3/users/ceda9da793df4350a9c002ca2c785843` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:27.738287 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc92e13-f49b-4ee0-8291-e780cc03a1fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=ceda9da793df4350a9c002ca2c785843)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:27.741037 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc92e13-f49b-4ee0-8291-e780cc03a1fa admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:27.816228 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-4fc92e13-f49b-4ee0-8291-e780cc03a1fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ceda9da793df4350a9c002ca2c785843'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:27.836829 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-4fc92e13-f49b-4ee0-8291-e780cc03a1fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ceda9da793df4350a9c002ca2c785843'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:27.849846 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-4fc92e13-f49b-4ee0-8291-e780cc03a1fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ceda9da793df4350a9c002ca2c785843'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:27.850930 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-4fc92e13-f49b-4ee0-8291-e780cc03a1fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ceda9da793df4350a9c002ca2c785843'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:27.851557 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-4fc92e13-f49b-4ee0-8291-e780cc03a1fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ceda9da793df4350a9c002ca2c785843'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:27.887416 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1022/2036] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 13:47:27 2022] DELETE /identity/v3/users/ceda9da793df4350a9c002ca2c785843 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:47:27.892902 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b98507-a7ba-4ac7-bd80-fa3e22863635 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:27.893861 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b98507-a7ba-4ac7-bd80-fa3e22863635 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:27.894151 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b98507-a7ba-4ac7-bd80-fa3e22863635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:27.918534 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-35b98507-a7ba-4ac7-bd80-fa3e22863635 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:47:27.921434 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-35b98507-a7ba-4ac7-bd80-fa3e22863635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:27.997146 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1015/2037] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:47:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:47:28.009080 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc92e13-f49b-4ee0-8291-e780cc03a1fa admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:28.010576 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-4fc92e13-f49b-4ee0-8291-e780cc03a1fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:28.060034 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:47:28.073319 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc92e13-f49b-4ee0-8291-e780cc03a1fa admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:28.078111 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02d4f4e-00e1-46d7-a179-7fc4d73ddb07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:28.078730 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02d4f4e-00e1-46d7-a179-7fc4d73ddb07 service neutron] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:28.079006 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02d4f4e-00e1-46d7-a179-7fc4d73ddb07 service neutron] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:28.079277 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02d4f4e-00e1-46d7-a179-7fc4d73ddb07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:28.088879 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02d4f4e-00e1-46d7-a179-7fc4d73ddb07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:28.090363 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02d4f4e-00e1-46d7-a179-7fc4d73ddb07 service neutron] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:28.101191 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1023/2038] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:47:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:47:28.409026 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b98507-a7ba-4ac7-bd80-fa3e22863635 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:28.410663 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-35b98507-a7ba-4ac7-bd80-fa3e22863635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:28.423912 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b98507-a7ba-4ac7-bd80-fa3e22863635 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:28.428721 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaa3d10-786c-4ecd-b3c9-ad0a37633a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:28.429848 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaa3d10-786c-4ecd-b3c9-ad0a37633a05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:28.430138 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaa3d10-786c-4ecd-b3c9-ad0a37633a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:28.430411 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaa3d10-786c-4ecd-b3c9-ad0a37633a05 admin admin] PATH_INFO: `/v3/projects/615000c9b213489bb17bc95eb475006f` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:28.433779 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaa3d10-786c-4ecd-b3c9-ad0a37633a05 admin admin] RBAC: Authorizing `identity:delete_project(project_id=615000c9b213489bb17bc95eb475006f)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:28.436063 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaa3d10-786c-4ecd-b3c9-ad0a37633a05 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:28.481834 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-abaa3d10-786c-4ecd-b3c9-ad0a37633a05 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '615000c9b213489bb17bc95eb475006f', 'request_id': 'req-abaa3d10-786c-4ecd-b3c9-ad0a37633a05'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:28.488248 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-abaa3d10-786c-4ecd-b3c9-ad0a37633a05 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '615000c9b213489bb17bc95eb475006f', 'request_id': 'req-abaa3d10-786c-4ecd-b3c9-ad0a37633a05'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:28.493740 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-abaa3d10-786c-4ecd-b3c9-ad0a37633a05 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '615000c9b213489bb17bc95eb475006f', 'request_id': 'req-abaa3d10-786c-4ecd-b3c9-ad0a37633a05'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:28.506036 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-abaa3d10-786c-4ecd-b3c9-ad0a37633a05 admin admin] The token cache is being invalidate because project 615000c9b213489bb17bc95eb475006f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 13:47:28.506322 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-abaa3d10-786c-4ecd-b3c9-ad0a37633a05 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:28.506946 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-abaa3d10-786c-4ecd-b3c9-ad0a37633a05 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:28.508437 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1016/2039] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 13:47:28 2022] DELETE /identity/v3/projects/615000c9b213489bb17bc95eb475006f => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:47:28.512405 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02d4f4e-00e1-46d7-a179-7fc4d73ddb07 service neutron] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:28.521843 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02d4f4e-00e1-46d7-a179-7fc4d73ddb07 service neutron] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:28.576451 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c98f54e-dc84-4b08-b4aa-4240f17ffb90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:28.576888 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c98f54e-dc84-4b08-b4aa-4240f17ffb90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:28.577170 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c98f54e-dc84-4b08-b4aa-4240f17ffb90 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:28.577444 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c98f54e-dc84-4b08-b4aa-4240f17ffb90 admin admin] PATH_INFO: `/v3/users/9da95b5111c4414e9b1830351f498a30` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:28.578646 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c98f54e-dc84-4b08-b4aa-4240f17ffb90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9da95b5111c4414e9b1830351f498a30)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:28.580622 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c98f54e-dc84-4b08-b4aa-4240f17ffb90 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:28.634088 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-5c98f54e-dc84-4b08-b4aa-4240f17ffb90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9da95b5111c4414e9b1830351f498a30'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:28.643434 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-5c98f54e-dc84-4b08-b4aa-4240f17ffb90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9da95b5111c4414e9b1830351f498a30'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:28.652054 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-5c98f54e-dc84-4b08-b4aa-4240f17ffb90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9da95b5111c4414e9b1830351f498a30'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:28.653091 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-5c98f54e-dc84-4b08-b4aa-4240f17ffb90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9da95b5111c4414e9b1830351f498a30'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:28.653744 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-5c98f54e-dc84-4b08-b4aa-4240f17ffb90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9da95b5111c4414e9b1830351f498a30'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:28.674821 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1024/2040] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 13:47:28 2022] DELETE /identity/v3/users/9da95b5111c4414e9b1830351f498a30 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:47:28.976274 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaa3d10-786c-4ecd-b3c9-ad0a37633a05 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:28.978272 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-abaa3d10-786c-4ecd-b3c9-ad0a37633a05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:29.020536 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaa3d10-786c-4ecd-b3c9-ad0a37633a05 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:29.061682 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:47:29.099881 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4a12fa-a059-43de-bba7-c42cb5ed74e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:29.100543 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4a12fa-a059-43de-bba7-c42cb5ed74e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:29.100931 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4a12fa-a059-43de-bba7-c42cb5ed74e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:29.101416 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4a12fa-a059-43de-bba7-c42cb5ed74e1 admin admin] PATH_INFO: `/v3/projects/c2cca54a1aeb45c4ab505f2ecea78549` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:29.103114 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4a12fa-a059-43de-bba7-c42cb5ed74e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2cca54a1aeb45c4ab505f2ecea78549)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:29.105427 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4a12fa-a059-43de-bba7-c42cb5ed74e1 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:29.163875 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-7f4a12fa-a059-43de-bba7-c42cb5ed74e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2cca54a1aeb45c4ab505f2ecea78549', 'request_id': 'req-7f4a12fa-a059-43de-bba7-c42cb5ed74e1'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:29.171071 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-7f4a12fa-a059-43de-bba7-c42cb5ed74e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2cca54a1aeb45c4ab505f2ecea78549', 'request_id': 'req-7f4a12fa-a059-43de-bba7-c42cb5ed74e1'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:29.176838 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-7f4a12fa-a059-43de-bba7-c42cb5ed74e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2cca54a1aeb45c4ab505f2ecea78549', 'request_id': 'req-7f4a12fa-a059-43de-bba7-c42cb5ed74e1'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:29.190102 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-7f4a12fa-a059-43de-bba7-c42cb5ed74e1 admin admin] The token cache is being invalidate because project c2cca54a1aeb45c4ab505f2ecea78549 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 13:47:29.190238 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-7f4a12fa-a059-43de-bba7-c42cb5ed74e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:29.190933 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-7f4a12fa-a059-43de-bba7-c42cb5ed74e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:29.192369 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1017/2041] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 13:47:28 2022] DELETE /identity/v3/projects/c2cca54a1aeb45c4ab505f2ecea78549 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:47:29.197328 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e81cbd-0e9d-4c0a-a3e5-b193b1c2f526 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:29.197629 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e81cbd-0e9d-4c0a-a3e5-b193b1c2f526 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:29.197906 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e81cbd-0e9d-4c0a-a3e5-b193b1c2f526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:29.219003 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-c1e81cbd-0e9d-4c0a-a3e5-b193b1c2f526 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:47:29.220242 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-c1e81cbd-0e9d-4c0a-a3e5-b193b1c2f526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:29.298749 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1025/2042] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:47:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:47:29.307363 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7191500a-743a-44b3-9644-2b3371804dfa None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:29.307756 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7191500a-743a-44b3-9644-2b3371804dfa None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:29.308106 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7191500a-743a-44b3-9644-2b3371804dfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:29.330718 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-7191500a-743a-44b3-9644-2b3371804dfa None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:47:29.331928 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-7191500a-743a-44b3-9644-2b3371804dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:29.340630 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1018/2043] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:47:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:47:29.346592 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e81cbd-0e9d-4c0a-a3e5-b193b1c2f526 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:29.356436 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e81cbd-0e9d-4c0a-a3e5-b193b1c2f526 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:29.361942 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2cacf8-6392-40d0-bafc-172af7fd7895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:29.362623 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2cacf8-6392-40d0-bafc-172af7fd7895 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:29.362970 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2cacf8-6392-40d0-bafc-172af7fd7895 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:29.363308 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2cacf8-6392-40d0-bafc-172af7fd7895 admin admin] PATH_INFO: `/v3/domains` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:29.364068 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2cacf8-6392-40d0-bafc-172af7fd7895 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:47:29.364458 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2cacf8-6392-40d0-bafc-172af7fd7895 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:29.366120 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2cacf8-6392-40d0-bafc-172af7fd7895 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:29.367665 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:47:29.383432 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1026/2044] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Jul 26 13:47:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:47:29.384510 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7191500a-743a-44b3-9644-2b3371804dfa None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:29.393566 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7191500a-743a-44b3-9644-2b3371804dfa None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:29.398493 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89850859-36e3-4db4-b006-7e989ea42c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:29.399098 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89850859-36e3-4db4-b006-7e989ea42c91 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:29.399376 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89850859-36e3-4db4-b006-7e989ea42c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:29.399667 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89850859-36e3-4db4-b006-7e989ea42c91 admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:29.400311 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89850859-36e3-4db4-b006-7e989ea42c91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:29.402227 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89850859-36e3-4db4-b006-7e989ea42c91 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:29.429799 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1019/2045] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 13:47:29 2022] POST /identity/v3/projects => generated 365 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 13:47:29.433902 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2cacf8-6392-40d0-bafc-172af7fd7895 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:29.444962 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2cacf8-6392-40d0-bafc-172af7fd7895 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:29.500152 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d153b44-9cea-44ee-9527-364bd9900ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:29.500701 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d153b44-9cea-44ee-9527-364bd9900ffe admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:29.500984 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d153b44-9cea-44ee-9527-364bd9900ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:29.501257 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d153b44-9cea-44ee-9527-364bd9900ffe admin admin] PATH_INFO: `/v3/users` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:29.501954 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d153b44-9cea-44ee-9527-364bd9900ffe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:29.503843 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d153b44-9cea-44ee-9527-364bd9900ffe admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:29.523674 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1027/2046] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 13:47:29 2022] POST /identity/v3/users => generated 283 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:47:29.527823 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89850859-36e3-4db4-b006-7e989ea42c91 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:29.538853 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89850859-36e3-4db4-b006-7e989ea42c91 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:29.543428 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380eb662-e1da-435e-950c-3754a0fb7cb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:29.544031 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380eb662-e1da-435e-950c-3754a0fb7cb3 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:29.544314 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380eb662-e1da-435e-950c-3754a0fb7cb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:29.544583 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380eb662-e1da-435e-950c-3754a0fb7cb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:29.545267 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380eb662-e1da-435e-950c-3754a0fb7cb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:29.547505 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380eb662-e1da-435e-950c-3754a0fb7cb3 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:29.557290 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1020/2047] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:47:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:47:29.561305 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d153b44-9cea-44ee-9527-364bd9900ffe admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:29.571183 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d153b44-9cea-44ee-9527-364bd9900ffe admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:29.575837 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d393c016-9bae-4b84-8d5a-da43d457e370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:29.576442 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d393c016-9bae-4b84-8d5a-da43d457e370 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:29.576722 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d393c016-9bae-4b84-8d5a-da43d457e370 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:29.577000 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d393c016-9bae-4b84-8d5a-da43d457e370 admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:29.577723 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d393c016-9bae-4b84-8d5a-da43d457e370 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:29.579290 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d393c016-9bae-4b84-8d5a-da43d457e370 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:29.589742 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1028/2048] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:47:29 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:47:29.594251 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380eb662-e1da-435e-950c-3754a0fb7cb3 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:29.603994 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380eb662-e1da-435e-950c-3754a0fb7cb3 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:29.609078 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feb9786-249c-40a0-b8bf-59bd4411b063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:29.609842 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feb9786-249c-40a0-b8bf-59bd4411b063 admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:29.610125 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feb9786-249c-40a0-b8bf-59bd4411b063 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:29.610396 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feb9786-249c-40a0-b8bf-59bd4411b063 admin admin] PATH_INFO: `/v3/projects/b708762c50314fa89fc955a4eff2ab99/users/26a3ea88c36c42048548a8d46804d3c8/roles/759f389b221049c4b01faf3b984cd8d7` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:29.624537 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6feb9786-249c-40a0-b8bf-59bd4411b063 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b708762c50314fa89fc955a4eff2ab99, user_id=26a3ea88c36c42048548a8d46804d3c8, role_id=759f389b221049c4b01faf3b984cd8d7)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:29.627185 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6feb9786-249c-40a0-b8bf-59bd4411b063 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:29.639861 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1021/2049] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 13:47:29 2022] PUT /identity/v3/projects/b708762c50314fa89fc955a4eff2ab99/users/26a3ea88c36c42048548a8d46804d3c8/roles/759f389b221049c4b01faf3b984cd8d7 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:47:29.649923 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72014dc5-4d8e-497f-bf39-5ac1f0d246b8 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:29.650283 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72014dc5-4d8e-497f-bf39-5ac1f0d246b8 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:29.650634 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72014dc5-4d8e-497f-bf39-5ac1f0d246b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:29.678510 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-72014dc5-4d8e-497f-bf39-5ac1f0d246b8 None None] MFA Rules not processed for user `26a3ea88c36c42048548a8d46804d3c8`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:47:29.679710 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-72014dc5-4d8e-497f-bf39-5ac1f0d246b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:29.740785 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1029/2050] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:47:29 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:47:29.752180 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feb9786-249c-40a0-b8bf-59bd4411b063 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:29.753819 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-6feb9786-249c-40a0-b8bf-59bd4411b063 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:29.813598 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feb9786-249c-40a0-b8bf-59bd4411b063 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:29.817211 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08005947-7762-41d4-b9bb-7f4e1e0beda3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:29.817868 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08005947-7762-41d4-b9bb-7f4e1e0beda3 service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:29.818158 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08005947-7762-41d4-b9bb-7f4e1e0beda3 service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:29.818431 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08005947-7762-41d4-b9bb-7f4e1e0beda3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:29.827333 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08005947-7762-41d4-b9bb-7f4e1e0beda3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:29.828784 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08005947-7762-41d4-b9bb-7f4e1e0beda3 service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:29.834077 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:47:29.841209 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1022/2051] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:47:29 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:47:33.560166 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72014dc5-4d8e-497f-bf39-5ac1f0d246b8 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:33.562310 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-72014dc5-4d8e-497f-bf39-5ac1f0d246b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:33.673297 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72014dc5-4d8e-497f-bf39-5ac1f0d246b8 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:33.680782 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6ad442-e9f7-48e6-98c9-d9e15a381e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:33.681736 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6ad442-e9f7-48e6-98c9-d9e15a381e3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:33.684127 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6ad442-e9f7-48e6-98c9-d9e15a381e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:33.684540 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6ad442-e9f7-48e6-98c9-d9e15a381e3d admin admin] PATH_INFO: `/v3/users/0291c8ea030942558cb1012c0f427e6d` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:33.687090 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6ad442-e9f7-48e6-98c9-d9e15a381e3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=0291c8ea030942558cb1012c0f427e6d)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:33.691098 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6ad442-e9f7-48e6-98c9-d9e15a381e3d admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:33.695452 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:47:33.788119 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-5b6ad442-e9f7-48e6-98c9-d9e15a381e3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0291c8ea030942558cb1012c0f427e6d'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:33.802011 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-5b6ad442-e9f7-48e6-98c9-d9e15a381e3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0291c8ea030942558cb1012c0f427e6d'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:33.817384 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-5b6ad442-e9f7-48e6-98c9-d9e15a381e3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0291c8ea030942558cb1012c0f427e6d'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:33.819698 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-5b6ad442-e9f7-48e6-98c9-d9e15a381e3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0291c8ea030942558cb1012c0f427e6d'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:33.821942 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-5b6ad442-e9f7-48e6-98c9-d9e15a381e3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0291c8ea030942558cb1012c0f427e6d'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:33.875116 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1030/2052] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 13:47:33 2022] DELETE /identity/v3/users/0291c8ea030942558cb1012c0f427e6d => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:47:33.880223 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cae8529-9558-411b-8db7-ac5aaa7f6b9f None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:33.881697 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cae8529-9558-411b-8db7-ac5aaa7f6b9f None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:33.883251 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cae8529-9558-411b-8db7-ac5aaa7f6b9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:33.926450 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-2cae8529-9558-411b-8db7-ac5aaa7f6b9f None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:47:33.927801 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-2cae8529-9558-411b-8db7-ac5aaa7f6b9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:34.009240 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1023/2053] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:47:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:47:34.019515 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6ad442-e9f7-48e6-98c9-d9e15a381e3d admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:34.021124 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-5b6ad442-e9f7-48e6-98c9-d9e15a381e3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:34.110797 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6ad442-e9f7-48e6-98c9-d9e15a381e3d admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:34.114683 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa58be0c-e56a-4581-a2c6-4817ecc0e0f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:34.115971 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa58be0c-e56a-4581-a2c6-4817ecc0e0f0 service neutron] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:34.116360 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa58be0c-e56a-4581-a2c6-4817ecc0e0f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:34.116734 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa58be0c-e56a-4581-a2c6-4817ecc0e0f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:34.135076 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa58be0c-e56a-4581-a2c6-4817ecc0e0f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:34.139579 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa58be0c-e56a-4581-a2c6-4817ecc0e0f0 service neutron] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:34.163099 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1031/2054] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:47:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 13:47:34.600082 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cae8529-9558-411b-8db7-ac5aaa7f6b9f None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:34.602226 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-2cae8529-9558-411b-8db7-ac5aaa7f6b9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:34.639316 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cae8529-9558-411b-8db7-ac5aaa7f6b9f None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:34.656859 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8aa152-a5bb-4a44-b43d-33ed4961a1d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:34.658264 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8aa152-a5bb-4a44-b43d-33ed4961a1d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:34.658619 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8aa152-a5bb-4a44-b43d-33ed4961a1d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:34.658961 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8aa152-a5bb-4a44-b43d-33ed4961a1d8 admin admin] PATH_INFO: `/v3/projects/6b3dd4e5489c47079f039821127998d6` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:34.660252 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8aa152-a5bb-4a44-b43d-33ed4961a1d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b3dd4e5489c47079f039821127998d6)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:34.662764 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8aa152-a5bb-4a44-b43d-33ed4961a1d8 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:34.731460 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-4a8aa152-a5bb-4a44-b43d-33ed4961a1d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b3dd4e5489c47079f039821127998d6', 'request_id': 'req-4a8aa152-a5bb-4a44-b43d-33ed4961a1d8'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:34.738535 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-4a8aa152-a5bb-4a44-b43d-33ed4961a1d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b3dd4e5489c47079f039821127998d6', 'request_id': 'req-4a8aa152-a5bb-4a44-b43d-33ed4961a1d8'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:34.744398 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-4a8aa152-a5bb-4a44-b43d-33ed4961a1d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b3dd4e5489c47079f039821127998d6', 'request_id': 'req-4a8aa152-a5bb-4a44-b43d-33ed4961a1d8'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:34.758790 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-4a8aa152-a5bb-4a44-b43d-33ed4961a1d8 admin admin] The token cache is being invalidate because project 6b3dd4e5489c47079f039821127998d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 13:47:34.759197 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-4a8aa152-a5bb-4a44-b43d-33ed4961a1d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:34.760089 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-4a8aa152-a5bb-4a44-b43d-33ed4961a1d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:34.765511 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1024/2055] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 13:47:34 2022] DELETE /identity/v3/projects/6b3dd4e5489c47079f039821127998d6 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:47:34.768406 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6d9e03-1f18-47b3-9b5d-bfa324b87519 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:34.769141 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6d9e03-1f18-47b3-9b5d-bfa324b87519 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:34.769141 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6d9e03-1f18-47b3-9b5d-bfa324b87519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:34.808436 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-0e6d9e03-1f18-47b3-9b5d-bfa324b87519 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:47:34.811607 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-0e6d9e03-1f18-47b3-9b5d-bfa324b87519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:34.893656 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1032/2056] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:47:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 13:47:34.901463 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66dfe3a-ccaf-422e-b3e1-f86b8f6fe576 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:34.901937 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66dfe3a-ccaf-422e-b3e1-f86b8f6fe576 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:34.901937 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66dfe3a-ccaf-422e-b3e1-f86b8f6fe576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:34.966727 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-a66dfe3a-ccaf-422e-b3e1-f86b8f6fe576 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:47:34.968737 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-a66dfe3a-ccaf-422e-b3e1-f86b8f6fe576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:34.981134 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1025/2057] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:47:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:47:34.986202 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6d9e03-1f18-47b3-9b5d-bfa324b87519 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:34.998352 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6d9e03-1f18-47b3-9b5d-bfa324b87519 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:35.007782 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427e9abf-38ec-48b3-872b-98f6fcea837d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:35.008415 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e9abf-38ec-48b3-872b-98f6fcea837d admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:35.008701 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e9abf-38ec-48b3-872b-98f6fcea837d admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:35.008977 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e9abf-38ec-48b3-872b-98f6fcea837d admin admin] PATH_INFO: `/v3/domains` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:35.009724 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427e9abf-38ec-48b3-872b-98f6fcea837d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:47:35.010033 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427e9abf-38ec-48b3-872b-98f6fcea837d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:35.011643 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427e9abf-38ec-48b3-872b-98f6fcea837d admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:35.027732 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1033/2058] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Jul 26 13:47:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:47:35.028906 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66dfe3a-ccaf-422e-b3e1-f86b8f6fe576 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:35.039620 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66dfe3a-ccaf-422e-b3e1-f86b8f6fe576 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:35.047167 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541a6cc0-9245-4560-a09e-556cefadb58e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:35.047804 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541a6cc0-9245-4560-a09e-556cefadb58e admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:35.048089 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541a6cc0-9245-4560-a09e-556cefadb58e admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:35.049529 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541a6cc0-9245-4560-a09e-556cefadb58e admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:35.049529 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541a6cc0-9245-4560-a09e-556cefadb58e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:35.050957 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541a6cc0-9245-4560-a09e-556cefadb58e admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:35.083056 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1026/2059] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 13:47:35 2022] POST /identity/v3/projects => generated 379 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:47:35.087286 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427e9abf-38ec-48b3-872b-98f6fcea837d admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:35.099099 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427e9abf-38ec-48b3-872b-98f6fcea837d admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:35.107849 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:47:35.183337 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32adcbd3-c7c7-453a-afb4-e00f9a727e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:35.183999 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32adcbd3-c7c7-453a-afb4-e00f9a727e63 admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:35.184286 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32adcbd3-c7c7-453a-afb4-e00f9a727e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:35.184617 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32adcbd3-c7c7-453a-afb4-e00f9a727e63 admin admin] PATH_INFO: `/v3/users` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:35.185216 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32adcbd3-c7c7-453a-afb4-e00f9a727e63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:35.188514 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32adcbd3-c7c7-453a-afb4-e00f9a727e63 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:35.219671 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1034/2060] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 13:47:35 2022] POST /identity/v3/users => generated 290 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 13:47:35.220990 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541a6cc0-9245-4560-a09e-556cefadb58e admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:35.234148 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541a6cc0-9245-4560-a09e-556cefadb58e admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:35.241228 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b8c14a-614c-4d61-a702-f06089feed94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:35.241964 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b8c14a-614c-4d61-a702-f06089feed94 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:35.242249 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b8c14a-614c-4d61-a702-f06089feed94 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:35.242534 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b8c14a-614c-4d61-a702-f06089feed94 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:35.243232 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b8c14a-614c-4d61-a702-f06089feed94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:35.245691 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b8c14a-614c-4d61-a702-f06089feed94 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:35.260834 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1027/2061] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:47:35 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:47:35.270969 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32adcbd3-c7c7-453a-afb4-e00f9a727e63 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:35.287774 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32adcbd3-c7c7-453a-afb4-e00f9a727e63 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:35.294408 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5cc712-0bd2-4c43-89b5-c81b8a9c0b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:35.295746 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5cc712-0bd2-4c43-89b5-c81b8a9c0b12 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:35.296287 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5cc712-0bd2-4c43-89b5-c81b8a9c0b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:35.297041 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5cc712-0bd2-4c43-89b5-c81b8a9c0b12 admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:35.297960 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5cc712-0bd2-4c43-89b5-c81b8a9c0b12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:35.299868 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5cc712-0bd2-4c43-89b5-c81b8a9c0b12 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:35.325180 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b8c14a-614c-4d61-a702-f06089feed94 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:35.325881 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1035/2062] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:47:35 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:47:35.340670 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b8c14a-614c-4d61-a702-f06089feed94 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:35.350623 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4637436f-2696-409d-adce-b901c7f91fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:35.352365 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4637436f-2696-409d-adce-b901c7f91fab admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:35.353130 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4637436f-2696-409d-adce-b901c7f91fab admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:35.353566 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4637436f-2696-409d-adce-b901c7f91fab admin admin] PATH_INFO: `/v3/projects/cb8ac2f54241495f9c55bfa75d288c52/users/22b8c6b94b8741b085081ca02e5316e6/roles/759f389b221049c4b01faf3b984cd8d7` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:35.377135 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4637436f-2696-409d-adce-b901c7f91fab admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb8ac2f54241495f9c55bfa75d288c52, user_id=22b8c6b94b8741b085081ca02e5316e6, role_id=759f389b221049c4b01faf3b984cd8d7)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:35.380324 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4637436f-2696-409d-adce-b901c7f91fab admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:35.407681 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cafd8e-017d-4288-991b-d8c165d92de9 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:35.407866 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cafd8e-017d-4288-991b-d8c165d92de9 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:35.408708 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cafd8e-017d-4288-991b-d8c165d92de9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:35.416982 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1028/2063] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 13:47:35 2022] PUT /identity/v3/projects/cb8ac2f54241495f9c55bfa75d288c52/users/22b8c6b94b8741b085081ca02e5316e6/roles/759f389b221049c4b01faf3b984cd8d7 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:47:35.447254 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-e8cafd8e-017d-4288-991b-d8c165d92de9 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:47:35.449576 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-e8cafd8e-017d-4288-991b-d8c165d92de9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:35.541748 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1036/2064] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:47:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 13:47:35.580743 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4637436f-2696-409d-adce-b901c7f91fab admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:35.583295 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-4637436f-2696-409d-adce-b901c7f91fab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:35.614515 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:47:35.704220 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4637436f-2696-409d-adce-b901c7f91fab admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:35.716716 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360832f8-2303-4087-8df4-269ce08132f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:35.718247 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360832f8-2303-4087-8df4-269ce08132f2 service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:35.718820 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360832f8-2303-4087-8df4-269ce08132f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:35.720178 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360832f8-2303-4087-8df4-269ce08132f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:35.753033 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360832f8-2303-4087-8df4-269ce08132f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:35.755249 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360832f8-2303-4087-8df4-269ce08132f2 service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:35.766856 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1029/2065] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:47:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:47:36.554113 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cafd8e-017d-4288-991b-d8c165d92de9 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:36.556717 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-e8cafd8e-017d-4288-991b-d8c165d92de9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:36.572151 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cafd8e-017d-4288-991b-d8c165d92de9 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:36.607674 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261ca054-d9e6-4b6f-a3fb-6368f311fe51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:36.608774 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261ca054-d9e6-4b6f-a3fb-6368f311fe51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:36.609059 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261ca054-d9e6-4b6f-a3fb-6368f311fe51 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:36.609547 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261ca054-d9e6-4b6f-a3fb-6368f311fe51 admin admin] PATH_INFO: `/v3/users/d24e49566a4248c1ba8d8db011bdd603` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:36.611283 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261ca054-d9e6-4b6f-a3fb-6368f311fe51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d24e49566a4248c1ba8d8db011bdd603)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:36.618819 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261ca054-d9e6-4b6f-a3fb-6368f311fe51 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:36.626061 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b45f92-3c9b-4657-8a38-fb5fdb366f0f None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:36.626972 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b45f92-3c9b-4657-8a38-fb5fdb366f0f None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:36.627419 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b45f92-3c9b-4657-8a38-fb5fdb366f0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:36.702127 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-b8b45f92-3c9b-4657-8a38-fb5fdb366f0f None None] MFA Rules not processed for user `22b8c6b94b8741b085081ca02e5316e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:47:36.707446 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-b8b45f92-3c9b-4657-8a38-fb5fdb366f0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:36.743471 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-261ca054-d9e6-4b6f-a3fb-6368f311fe51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd24e49566a4248c1ba8d8db011bdd603'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:36.776667 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-261ca054-d9e6-4b6f-a3fb-6368f311fe51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd24e49566a4248c1ba8d8db011bdd603'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:36.789529 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-261ca054-d9e6-4b6f-a3fb-6368f311fe51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd24e49566a4248c1ba8d8db011bdd603'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:36.794495 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-261ca054-d9e6-4b6f-a3fb-6368f311fe51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd24e49566a4248c1ba8d8db011bdd603'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:36.797368 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-261ca054-d9e6-4b6f-a3fb-6368f311fe51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd24e49566a4248c1ba8d8db011bdd603'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:36.860696 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1030/2066] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:47:36 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:47:36.875455 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b45f92-3c9b-4657-8a38-fb5fdb366f0f None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:36.876214 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1037/2067] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 13:47:36 2022] DELETE /identity/v3/users/d24e49566a4248c1ba8d8db011bdd603 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:47:36.880007 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-b8b45f92-3c9b-4657-8a38-fb5fdb366f0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:36.885532 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab2df27-14bb-4e0a-b965-e5a6d856830e None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:36.885944 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab2df27-14bb-4e0a-b965-e5a6d856830e None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:36.886328 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab2df27-14bb-4e0a-b965-e5a6d856830e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:36.899078 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:47:36.916993 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-7ab2df27-14bb-4e0a-b965-e5a6d856830e None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:47:36.918418 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-7ab2df27-14bb-4e0a-b965-e5a6d856830e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:37.029567 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1038/2068] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:47:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:47:37.045988 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b45f92-3c9b-4657-8a38-fb5fdb366f0f None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:37.050366 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab2df27-14bb-4e0a-b965-e5a6d856830e None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:37.053718 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-7ab2df27-14bb-4e0a-b965-e5a6d856830e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:37.157866 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fedccd-0def-4533-9351-da3ed98c8817 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:37.158900 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fedccd-0def-4533-9351-da3ed98c8817 service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:37.159445 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fedccd-0def-4533-9351-da3ed98c8817 service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:37.159846 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fedccd-0def-4533-9351-da3ed98c8817 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:37.160967 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab2df27-14bb-4e0a-b965-e5a6d856830e None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:37.165959 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e9733b-3bf0-4ddd-972d-d008a2a1c711 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:37.166675 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e9733b-3bf0-4ddd-972d-d008a2a1c711 service neutron] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:37.167027 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e9733b-3bf0-4ddd-972d-d008a2a1c711 service neutron] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:37.167390 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e9733b-3bf0-4ddd-972d-d008a2a1c711 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:37.184157 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e9733b-3bf0-4ddd-972d-d008a2a1c711 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:37.186582 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e9733b-3bf0-4ddd-972d-d008a2a1c711 service neutron] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:37.192125 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:47:37.205099 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1039/2069] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:47:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:47:37.206466 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e9733b-3bf0-4ddd-972d-d008a2a1c711 service neutron] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:37.208036 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-14e9733b-3bf0-4ddd-972d-d008a2a1c711 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:37.224322 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e9733b-3bf0-4ddd-972d-d008a2a1c711 service neutron] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:37.230455 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5fbc34-7454-4b32-bb2b-c10ee8a82dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:37.232203 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5fbc34-7454-4b32-bb2b-c10ee8a82dc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:37.233147 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5fbc34-7454-4b32-bb2b-c10ee8a82dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:37.233723 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5fbc34-7454-4b32-bb2b-c10ee8a82dc7 admin admin] PATH_INFO: `/v3/users/9b6603042a734fe6a5ac7b62eb959090` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:37.235074 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5fbc34-7454-4b32-bb2b-c10ee8a82dc7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b6603042a734fe6a5ac7b62eb959090)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:37.237356 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5fbc34-7454-4b32-bb2b-c10ee8a82dc7 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:37.290460 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fedccd-0def-4533-9351-da3ed98c8817 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:37.293913 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fedccd-0def-4533-9351-da3ed98c8817 service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:37.327299 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1031/2070] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:47:36 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 13:47:37.338673 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-8a5fbc34-7454-4b32-bb2b-c10ee8a82dc7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b6603042a734fe6a5ac7b62eb959090'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:37.357010 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-8a5fbc34-7454-4b32-bb2b-c10ee8a82dc7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b6603042a734fe6a5ac7b62eb959090'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:37.370098 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-8a5fbc34-7454-4b32-bb2b-c10ee8a82dc7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b6603042a734fe6a5ac7b62eb959090'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:37.370863 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-8a5fbc34-7454-4b32-bb2b-c10ee8a82dc7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b6603042a734fe6a5ac7b62eb959090'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:37.371500 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-8a5fbc34-7454-4b32-bb2b-c10ee8a82dc7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b6603042a734fe6a5ac7b62eb959090'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:37.406881 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1040/2071] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 13:47:37 2022] DELETE /identity/v3/users/9b6603042a734fe6a5ac7b62eb959090 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:47:37.407648 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971eb526-87eb-49a0-94f5-faa8872665f8 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:37.407943 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971eb526-87eb-49a0-94f5-faa8872665f8 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:37.408224 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971eb526-87eb-49a0-94f5-faa8872665f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:37.450190 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-971eb526-87eb-49a0-94f5-faa8872665f8 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:47:37.453279 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-971eb526-87eb-49a0-94f5-faa8872665f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:37.554330 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1032/2072] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:47:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 13:47:37.565031 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5fbc34-7454-4b32-bb2b-c10ee8a82dc7 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:37.568376 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-8a5fbc34-7454-4b32-bb2b-c10ee8a82dc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:37.594484 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:47:37.616680 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971eb526-87eb-49a0-94f5-faa8872665f8 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:37.618134 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-971eb526-87eb-49a0-94f5-faa8872665f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:37.638905 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971eb526-87eb-49a0-94f5-faa8872665f8 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:37.641111 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5fbc34-7454-4b32-bb2b-c10ee8a82dc7 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:37.645124 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538298c0-8278-45ec-bfa8-19e3a70470fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:37.645949 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538298c0-8278-45ec-bfa8-19e3a70470fd service neutron] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:37.646165 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8c6a72-88f9-471c-9700-578c684219a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:37.646363 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538298c0-8278-45ec-bfa8-19e3a70470fd service neutron] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:37.646710 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538298c0-8278-45ec-bfa8-19e3a70470fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:37.646940 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8c6a72-88f9-471c-9700-578c684219a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:37.647343 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8c6a72-88f9-471c-9700-578c684219a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:37.647629 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8c6a72-88f9-471c-9700-578c684219a9 admin admin] PATH_INFO: `/v3/projects/8071f81ccad0427db5883efba1c28b0b` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:37.649121 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8c6a72-88f9-471c-9700-578c684219a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8071f81ccad0427db5883efba1c28b0b)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:37.651419 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8c6a72-88f9-471c-9700-578c684219a9 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:37.659788 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538298c0-8278-45ec-bfa8-19e3a70470fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:37.661889 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538298c0-8278-45ec-bfa8-19e3a70470fd service neutron] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:37.673862 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1041/2073] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:47:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:47:37.724970 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-6f8c6a72-88f9-471c-9700-578c684219a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8071f81ccad0427db5883efba1c28b0b', 'request_id': 'req-6f8c6a72-88f9-471c-9700-578c684219a9'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:37.733298 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-6f8c6a72-88f9-471c-9700-578c684219a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8071f81ccad0427db5883efba1c28b0b', 'request_id': 'req-6f8c6a72-88f9-471c-9700-578c684219a9'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:37.751313 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-6f8c6a72-88f9-471c-9700-578c684219a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8071f81ccad0427db5883efba1c28b0b', 'request_id': 'req-6f8c6a72-88f9-471c-9700-578c684219a9'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:37.776471 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-6f8c6a72-88f9-471c-9700-578c684219a9 admin admin] The token cache is being invalidate because project 8071f81ccad0427db5883efba1c28b0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 13:47:37.776862 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-6f8c6a72-88f9-471c-9700-578c684219a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:37.777626 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-6f8c6a72-88f9-471c-9700-578c684219a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:37.787575 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1033/2074] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 13:47:37 2022] DELETE /identity/v3/projects/8071f81ccad0427db5883efba1c28b0b => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:47:37.793954 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538298c0-8278-45ec-bfa8-19e3a70470fd service neutron] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:37.811351 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538298c0-8278-45ec-bfa8-19e3a70470fd service neutron] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:37.911336 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf5ed1c-1738-419a-bc4d-f9aca06caba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:37.912007 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf5ed1c-1738-419a-bc4d-f9aca06caba0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:37.912287 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf5ed1c-1738-419a-bc4d-f9aca06caba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:37.912563 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf5ed1c-1738-419a-bc4d-f9aca06caba0 admin admin] PATH_INFO: `/v3/users/e915a05773e34697a7c0092e09520736` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:37.913963 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf5ed1c-1738-419a-bc4d-f9aca06caba0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e915a05773e34697a7c0092e09520736)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:37.916287 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf5ed1c-1738-419a-bc4d-f9aca06caba0 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:37.985596 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-3bf5ed1c-1738-419a-bc4d-f9aca06caba0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e915a05773e34697a7c0092e09520736'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:37.995443 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-3bf5ed1c-1738-419a-bc4d-f9aca06caba0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e915a05773e34697a7c0092e09520736'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:38.004291 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-3bf5ed1c-1738-419a-bc4d-f9aca06caba0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e915a05773e34697a7c0092e09520736'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:38.005061 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-3bf5ed1c-1738-419a-bc4d-f9aca06caba0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e915a05773e34697a7c0092e09520736'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:38.006100 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-3bf5ed1c-1738-419a-bc4d-f9aca06caba0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e915a05773e34697a7c0092e09520736'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:38.039801 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1042/2075] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 13:47:37 2022] DELETE /identity/v3/users/e915a05773e34697a7c0092e09520736 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:47:38.059189 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8c6a72-88f9-471c-9700-578c684219a9 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:38.061181 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-6f8c6a72-88f9-471c-9700-578c684219a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:38.109322 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8c6a72-88f9-471c-9700-578c684219a9 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:38.179004 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:47:38.195105 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd74f960-d0b2-4acb-b13c-01a43fa550f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:38.195929 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd74f960-d0b2-4acb-b13c-01a43fa550f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:38.196318 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd74f960-d0b2-4acb-b13c-01a43fa550f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:38.196742 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd74f960-d0b2-4acb-b13c-01a43fa550f5 admin admin] PATH_INFO: `/v3/projects/0f46e4161fd14df8a2c86857b20a6d2d` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:38.198339 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd74f960-d0b2-4acb-b13c-01a43fa550f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f46e4161fd14df8a2c86857b20a6d2d)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:38.200466 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd74f960-d0b2-4acb-b13c-01a43fa550f5 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:38.267349 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-dd74f960-d0b2-4acb-b13c-01a43fa550f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f46e4161fd14df8a2c86857b20a6d2d', 'request_id': 'req-dd74f960-d0b2-4acb-b13c-01a43fa550f5'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:38.274581 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-dd74f960-d0b2-4acb-b13c-01a43fa550f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f46e4161fd14df8a2c86857b20a6d2d', 'request_id': 'req-dd74f960-d0b2-4acb-b13c-01a43fa550f5'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:38.281887 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-dd74f960-d0b2-4acb-b13c-01a43fa550f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f46e4161fd14df8a2c86857b20a6d2d', 'request_id': 'req-dd74f960-d0b2-4acb-b13c-01a43fa550f5'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:38.295445 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-dd74f960-d0b2-4acb-b13c-01a43fa550f5 admin admin] The token cache is being invalidate because project 0f46e4161fd14df8a2c86857b20a6d2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 13:47:38.295835 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-dd74f960-d0b2-4acb-b13c-01a43fa550f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:38.296563 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-dd74f960-d0b2-4acb-b13c-01a43fa550f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:38.298338 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1034/2076] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 13:47:38 2022] DELETE /identity/v3/projects/0f46e4161fd14df8a2c86857b20a6d2d => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:47:38.642853 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf5ed1c-1738-419a-bc4d-f9aca06caba0 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:38.644342 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-3bf5ed1c-1738-419a-bc4d-f9aca06caba0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:38.658498 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf5ed1c-1738-419a-bc4d-f9aca06caba0 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:38.697372 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:47:38.717877 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c03fba-b2d2-494e-a3ee-8c98c306ad19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:38.718516 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c03fba-b2d2-494e-a3ee-8c98c306ad19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:38.718796 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c03fba-b2d2-494e-a3ee-8c98c306ad19 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:38.719070 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c03fba-b2d2-494e-a3ee-8c98c306ad19 admin admin] PATH_INFO: `/v3/projects/5b0bc0916d8845bbb8fa2983ecadf587` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:38.720209 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c03fba-b2d2-494e-a3ee-8c98c306ad19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b0bc0916d8845bbb8fa2983ecadf587)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:38.722129 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c03fba-b2d2-494e-a3ee-8c98c306ad19 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:38.766951 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-90c03fba-b2d2-494e-a3ee-8c98c306ad19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b0bc0916d8845bbb8fa2983ecadf587', 'request_id': 'req-90c03fba-b2d2-494e-a3ee-8c98c306ad19'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:38.773896 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-90c03fba-b2d2-494e-a3ee-8c98c306ad19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b0bc0916d8845bbb8fa2983ecadf587', 'request_id': 'req-90c03fba-b2d2-494e-a3ee-8c98c306ad19'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:38.780693 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-90c03fba-b2d2-494e-a3ee-8c98c306ad19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b0bc0916d8845bbb8fa2983ecadf587', 'request_id': 'req-90c03fba-b2d2-494e-a3ee-8c98c306ad19'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:38.792682 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-90c03fba-b2d2-494e-a3ee-8c98c306ad19 admin admin] The token cache is being invalidate because project 5b0bc0916d8845bbb8fa2983ecadf587 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86933) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 13:47:38.792955 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-90c03fba-b2d2-494e-a3ee-8c98c306ad19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:38.793733 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-90c03fba-b2d2-494e-a3ee-8c98c306ad19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:38.796111 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1043/2077] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 13:47:38 2022] DELETE /identity/v3/projects/5b0bc0916d8845bbb8fa2983ecadf587 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:47:38.801309 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ba8255-aa4a-4480-8b6b-98cb967ff42c None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:38.801829 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ba8255-aa4a-4480-8b6b-98cb967ff42c None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:38.801922 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ba8255-aa4a-4480-8b6b-98cb967ff42c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:38.822369 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-b5ba8255-aa4a-4480-8b6b-98cb967ff42c None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:47:38.823573 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-b5ba8255-aa4a-4480-8b6b-98cb967ff42c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:38.836081 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60480707-e20e-49a1-a8c7-43b1ca5a62ee None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:38.836369 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60480707-e20e-49a1-a8c7-43b1ca5a62ee None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:38.836642 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60480707-e20e-49a1-a8c7-43b1ca5a62ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:38.855726 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-60480707-e20e-49a1-a8c7-43b1ca5a62ee None None] MFA Rules not processed for user `22b8c6b94b8741b085081ca02e5316e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:47:38.856932 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-60480707-e20e-49a1-a8c7-43b1ca5a62ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:38.883221 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1035/2078] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:47:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:47:38.892074 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae46232-0328-4c02-abae-0382b7df9e9e None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:38.892367 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae46232-0328-4c02-abae-0382b7df9e9e None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:38.892647 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae46232-0328-4c02-abae-0382b7df9e9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:38.912256 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-6ae46232-0328-4c02-abae-0382b7df9e9e None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:47:38.913128 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1044/2079] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:47:38 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 13:47:38.913459 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-6ae46232-0328-4c02-abae-0382b7df9e9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:38.921382 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1036/2080] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:47:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:47:38.925852 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60480707-e20e-49a1-a8c7-43b1ca5a62ee None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:38.936613 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60480707-e20e-49a1-a8c7-43b1ca5a62ee None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:38.941142 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e108061c-5be3-4f0f-945e-faee0a5242f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:38.941834 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e108061c-5be3-4f0f-945e-faee0a5242f3 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:38.942117 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e108061c-5be3-4f0f-945e-faee0a5242f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:38.942379 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e108061c-5be3-4f0f-945e-faee0a5242f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:38.943273 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e108061c-5be3-4f0f-945e-faee0a5242f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:47:38.943576 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e108061c-5be3-4f0f-945e-faee0a5242f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:38.945234 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e108061c-5be3-4f0f-945e-faee0a5242f3 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:38.958128 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1045/2081] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Jul 26 13:47:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:47:38.963059 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae46232-0328-4c02-abae-0382b7df9e9e None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:38.973732 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae46232-0328-4c02-abae-0382b7df9e9e None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:38.978785 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371c4c00-b3d2-4215-a4bd-d63ca4772e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:38.979403 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371c4c00-b3d2-4215-a4bd-d63ca4772e4c admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:38.979685 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371c4c00-b3d2-4215-a4bd-d63ca4772e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:38.979965 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371c4c00-b3d2-4215-a4bd-d63ca4772e4c admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:38.980629 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371c4c00-b3d2-4215-a4bd-d63ca4772e4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:38.982583 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371c4c00-b3d2-4215-a4bd-d63ca4772e4c admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:39.015830 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1037/2082] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 13:47:38 2022] POST /identity/v3/projects => generated 365 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:47:39.017064 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e108061c-5be3-4f0f-945e-faee0a5242f3 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:39.027069 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e108061c-5be3-4f0f-945e-faee0a5242f3 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:39.084541 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8139997d-ed65-49e7-940a-b2d8c0e7df53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:39.085166 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8139997d-ed65-49e7-940a-b2d8c0e7df53 admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:39.085447 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8139997d-ed65-49e7-940a-b2d8c0e7df53 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:39.085759 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8139997d-ed65-49e7-940a-b2d8c0e7df53 admin admin] PATH_INFO: `/v3/users` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:39.086422 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8139997d-ed65-49e7-940a-b2d8c0e7df53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:39.088332 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8139997d-ed65-49e7-940a-b2d8c0e7df53 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:39.097671 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:47:39.114545 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1046/2083] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 13:47:39 2022] POST /identity/v3/users => generated 283 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:47:39.115710 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371c4c00-b3d2-4215-a4bd-d63ca4772e4c admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:39.126399 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371c4c00-b3d2-4215-a4bd-d63ca4772e4c admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:39.131078 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ad050c-5ea3-4edd-9a5a-793209176ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:39.131690 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ad050c-5ea3-4edd-9a5a-793209176ec8 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:39.131970 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ad050c-5ea3-4edd-9a5a-793209176ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:39.132241 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ad050c-5ea3-4edd-9a5a-793209176ec8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:39.132935 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ad050c-5ea3-4edd-9a5a-793209176ec8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:39.134663 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ad050c-5ea3-4edd-9a5a-793209176ec8 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:39.146180 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1038/2084] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:47:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 13:47:39.149385 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8139997d-ed65-49e7-940a-b2d8c0e7df53 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:39.162779 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8139997d-ed65-49e7-940a-b2d8c0e7df53 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:39.168270 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98974ae1-8757-428c-a1b5-7b4e5bdba3d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:39.168895 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98974ae1-8757-428c-a1b5-7b4e5bdba3d5 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:39.169183 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98974ae1-8757-428c-a1b5-7b4e5bdba3d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:39.169639 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98974ae1-8757-428c-a1b5-7b4e5bdba3d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:39.170230 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98974ae1-8757-428c-a1b5-7b4e5bdba3d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:39.171810 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98974ae1-8757-428c-a1b5-7b4e5bdba3d5 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:39.184383 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1047/2085] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:47:39 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:47:39.188735 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ad050c-5ea3-4edd-9a5a-793209176ec8 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:39.199771 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ad050c-5ea3-4edd-9a5a-793209176ec8 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:39.205756 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa54113f-d8e3-480b-a1ae-ffdc257c81ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:39.206666 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa54113f-d8e3-480b-a1ae-ffdc257c81ea admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:39.207052 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa54113f-d8e3-480b-a1ae-ffdc257c81ea admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:39.207428 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa54113f-d8e3-480b-a1ae-ffdc257c81ea admin admin] PATH_INFO: `/v3/projects/cbfe6f72bde949d5a190688dbde4bc7d/users/612738f77b084d348756e8835c488051/roles/759f389b221049c4b01faf3b984cd8d7` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:39.227441 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa54113f-d8e3-480b-a1ae-ffdc257c81ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbfe6f72bde949d5a190688dbde4bc7d, user_id=612738f77b084d348756e8835c488051, role_id=759f389b221049c4b01faf3b984cd8d7)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:39.230632 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa54113f-d8e3-480b-a1ae-ffdc257c81ea admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:39.245862 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1039/2086] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 13:47:39 2022] PUT /identity/v3/projects/cbfe6f72bde949d5a190688dbde4bc7d/users/612738f77b084d348756e8835c488051/roles/759f389b221049c4b01faf3b984cd8d7 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 13:47:39.256454 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fe34e9-5f00-439e-97a4-ffabded27a3b None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:39.256744 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fe34e9-5f00-439e-97a4-ffabded27a3b None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:39.257018 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fe34e9-5f00-439e-97a4-ffabded27a3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:39.293874 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-d1fe34e9-5f00-439e-97a4-ffabded27a3b None None] MFA Rules not processed for user `612738f77b084d348756e8835c488051`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:47:39.295227 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-d1fe34e9-5f00-439e-97a4-ffabded27a3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:39.370156 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1048/2087] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:47:39 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:47:39.374986 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa54113f-d8e3-480b-a1ae-ffdc257c81ea admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:39.391455 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:47:39.460724 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa54113f-d8e3-480b-a1ae-ffdc257c81ea admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:39.465853 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d007faf6-b48a-4016-919f-fd62d9e5160a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:39.466484 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d007faf6-b48a-4016-919f-fd62d9e5160a admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:39.466815 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d007faf6-b48a-4016-919f-fd62d9e5160a admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:39.467311 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d007faf6-b48a-4016-919f-fd62d9e5160a admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:39.467990 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d007faf6-b48a-4016-919f-fd62d9e5160a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:39.469921 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d007faf6-b48a-4016-919f-fd62d9e5160a admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:39.504649 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1040/2088] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 13:47:39 2022] POST /identity/v3/projects => generated 365 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:47:39.507879 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fe34e9-5f00-439e-97a4-ffabded27a3b None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:39.521669 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fe34e9-5f00-439e-97a4-ffabded27a3b None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:39.599055 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4389d6c-6057-479e-b0e2-7a91c49b39ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:39.599685 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4389d6c-6057-479e-b0e2-7a91c49b39ae admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:39.600091 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4389d6c-6057-479e-b0e2-7a91c49b39ae admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:39.600374 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4389d6c-6057-479e-b0e2-7a91c49b39ae admin admin] PATH_INFO: `/v3/users` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:39.601048 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4389d6c-6057-479e-b0e2-7a91c49b39ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:39.603215 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4389d6c-6057-479e-b0e2-7a91c49b39ae admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:39.633251 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1049/2089] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 13:47:39 2022] POST /identity/v3/users => generated 289 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:47:39.637174 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d007faf6-b48a-4016-919f-fd62d9e5160a admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:39.657837 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d007faf6-b48a-4016-919f-fd62d9e5160a admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:39.663642 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdc2a37-e881-41a0-81f4-24f8061261ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:39.664381 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdc2a37-e881-41a0-81f4-24f8061261ac admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:39.664834 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdc2a37-e881-41a0-81f4-24f8061261ac admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:39.665316 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdc2a37-e881-41a0-81f4-24f8061261ac admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:39.666204 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdc2a37-e881-41a0-81f4-24f8061261ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:39.667948 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdc2a37-e881-41a0-81f4-24f8061261ac admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:39.683143 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1041/2090] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:47:39 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 13:47:39.687929 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4389d6c-6057-479e-b0e2-7a91c49b39ae admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:39.698451 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4389d6c-6057-479e-b0e2-7a91c49b39ae admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:39.707879 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206bcafb-6d67-431f-af4f-27447a8e39ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:39.708768 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206bcafb-6d67-431f-af4f-27447a8e39ea admin admin] REQUEST_METHOD: `PUT` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:39.709145 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206bcafb-6d67-431f-af4f-27447a8e39ea admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:39.709759 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206bcafb-6d67-431f-af4f-27447a8e39ea admin admin] PATH_INFO: `/v3/projects/3ffa008eaa95413e87f747a754a48606/users/50a617499a8b4dcf96b7d104958d33bb/roles/6e0201a9a6484f7d8b112b1eafff70bd` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:39.725457 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206bcafb-6d67-431f-af4f-27447a8e39ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ffa008eaa95413e87f747a754a48606, user_id=50a617499a8b4dcf96b7d104958d33bb, role_id=6e0201a9a6484f7d8b112b1eafff70bd)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:39.728356 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206bcafb-6d67-431f-af4f-27447a8e39ea admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:39.743462 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1050/2091] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 13:47:39 2022] PUT /identity/v3/projects/3ffa008eaa95413e87f747a754a48606/users/50a617499a8b4dcf96b7d104958d33bb/roles/6e0201a9a6484f7d8b112b1eafff70bd => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 13:47:39.747530 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdc2a37-e881-41a0-81f4-24f8061261ac admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:39.759355 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdc2a37-e881-41a0-81f4-24f8061261ac admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:39.828230 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee497a72-2f4e-4020-979f-96c4244d246c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:39.828843 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee497a72-2f4e-4020-979f-96c4244d246c admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:39.829124 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee497a72-2f4e-4020-979f-96c4244d246c admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:39.829397 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee497a72-2f4e-4020-979f-96c4244d246c admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:39.830164 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee497a72-2f4e-4020-979f-96c4244d246c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:39.831746 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee497a72-2f4e-4020-979f-96c4244d246c admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:39.846915 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1042/2092] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:47:39 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 13:47:39.847919 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206bcafb-6d67-431f-af4f-27447a8e39ea admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:39.859641 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206bcafb-6d67-431f-af4f-27447a8e39ea admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:39.865191 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601dce87-0151-4495-b12d-c4b64d7ddae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:39.866111 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601dce87-0151-4495-b12d-c4b64d7ddae4 admin admin] REQUEST_METHOD: `PUT` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:39.866402 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601dce87-0151-4495-b12d-c4b64d7ddae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:39.866678 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601dce87-0151-4495-b12d-c4b64d7ddae4 admin admin] PATH_INFO: `/v3/projects/3ffa008eaa95413e87f747a754a48606/users/50a617499a8b4dcf96b7d104958d33bb/roles/6e0201a9a6484f7d8b112b1eafff70bd` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:39.868849 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601dce87-0151-4495-b12d-c4b64d7ddae4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ffa008eaa95413e87f747a754a48606, user_id=50a617499a8b4dcf96b7d104958d33bb, role_id=6e0201a9a6484f7d8b112b1eafff70bd)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:39.871750 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601dce87-0151-4495-b12d-c4b64d7ddae4 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:39.882246 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1051/2093] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 13:47:39 2022] PUT /identity/v3/projects/3ffa008eaa95413e87f747a754a48606/users/50a617499a8b4dcf96b7d104958d33bb/roles/6e0201a9a6484f7d8b112b1eafff70bd => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:47:39.894629 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67f4ccd-2467-4c13-8ee7-6409fd6bdc64 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:39.894933 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67f4ccd-2467-4c13-8ee7-6409fd6bdc64 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:39.895230 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67f4ccd-2467-4c13-8ee7-6409fd6bdc64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:39.923314 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-c67f4ccd-2467-4c13-8ee7-6409fd6bdc64 None None] MFA Rules not processed for user `50a617499a8b4dcf96b7d104958d33bb`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:47:39.924555 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-c67f4ccd-2467-4c13-8ee7-6409fd6bdc64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:39.996088 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1043/2094] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:47:39 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:47:40.007294 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601dce87-0151-4495-b12d-c4b64d7ddae4 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:40.009015 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-601dce87-0151-4495-b12d-c4b64d7ddae4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:40.039698 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:47:40.076314 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601dce87-0151-4495-b12d-c4b64d7ddae4 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:40.080654 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193adac0-3ade-489b-8766-1fb5fdaec0c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:40.081280 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193adac0-3ade-489b-8766-1fb5fdaec0c9 service cinder] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:40.081611 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193adac0-3ade-489b-8766-1fb5fdaec0c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:40.081890 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193adac0-3ade-489b-8766-1fb5fdaec0c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:40.091118 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193adac0-3ade-489b-8766-1fb5fdaec0c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:40.092565 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193adac0-3ade-489b-8766-1fb5fdaec0c9 service cinder] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:40.102022 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1052/2095] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:47:40 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:47:40.180435 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67f4ccd-2467-4c13-8ee7-6409fd6bdc64 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:40.190650 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67f4ccd-2467-4c13-8ee7-6409fd6bdc64 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:40.194172 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43c3226-636f-4917-8025-e54697fcf865 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:40.194777 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43c3226-636f-4917-8025-e54697fcf865 service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:40.195056 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43c3226-636f-4917-8025-e54697fcf865 service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:40.195327 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43c3226-636f-4917-8025-e54697fcf865 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:40.200404 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:47:40.251541 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43c3226-636f-4917-8025-e54697fcf865 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:40.252954 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43c3226-636f-4917-8025-e54697fcf865 service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:40.261559 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1044/2096] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:47:40 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:47:45.098046 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193adac0-3ade-489b-8766-1fb5fdaec0c9 service cinder] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:45.170060 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193adac0-3ade-489b-8766-1fb5fdaec0c9 service cinder] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:45.175878 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081f0edf-33f3-459a-b6db-1b4cc90b80e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:45.176530 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081f0edf-33f3-459a-b6db-1b4cc90b80e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:45.176861 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081f0edf-33f3-459a-b6db-1b4cc90b80e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:45.177142 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081f0edf-33f3-459a-b6db-1b4cc90b80e7 admin admin] PATH_INFO: `/v3/users/612738f77b084d348756e8835c488051` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:45.178358 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081f0edf-33f3-459a-b6db-1b4cc90b80e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=612738f77b084d348756e8835c488051)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:45.180324 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081f0edf-33f3-459a-b6db-1b4cc90b80e7 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:45.241990 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-081f0edf-33f3-459a-b6db-1b4cc90b80e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '612738f77b084d348756e8835c488051'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:45.251528 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-081f0edf-33f3-459a-b6db-1b4cc90b80e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '612738f77b084d348756e8835c488051'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:45.259481 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-081f0edf-33f3-459a-b6db-1b4cc90b80e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '612738f77b084d348756e8835c488051'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:45.260294 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-081f0edf-33f3-459a-b6db-1b4cc90b80e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '612738f77b084d348756e8835c488051'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:45.261057 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-081f0edf-33f3-459a-b6db-1b4cc90b80e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '612738f77b084d348756e8835c488051'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:45.287973 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1053/2097] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 13:47:45 2022] DELETE /identity/v3/users/612738f77b084d348756e8835c488051 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:47:45.292682 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46ab991-4cb2-4c42-83bf-8db4db680cfa None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:45.293274 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46ab991-4cb2-4c42-83bf-8db4db680cfa None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:45.293606 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46ab991-4cb2-4c42-83bf-8db4db680cfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:45.318252 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-d46ab991-4cb2-4c42-83bf-8db4db680cfa None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:47:45.319585 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-d46ab991-4cb2-4c42-83bf-8db4db680cfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:45.429628 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1045/2098] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:47:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:47:45.441655 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081f0edf-33f3-459a-b6db-1b4cc90b80e7 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:45.443475 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-081f0edf-33f3-459a-b6db-1b4cc90b80e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:45.519485 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:47:45.534974 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081f0edf-33f3-459a-b6db-1b4cc90b80e7 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:45.540698 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3afc24b-d9fe-4a53-ae01-72c9c687edf3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:45.541401 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3afc24b-d9fe-4a53-ae01-72c9c687edf3 service neutron] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:45.541733 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3afc24b-d9fe-4a53-ae01-72c9c687edf3 service neutron] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:45.542011 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3afc24b-d9fe-4a53-ae01-72c9c687edf3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:45.555237 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3afc24b-d9fe-4a53-ae01-72c9c687edf3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:45.556761 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3afc24b-d9fe-4a53-ae01-72c9c687edf3 service neutron] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:45.567432 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1054/2099] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:47:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 13:47:46.034780 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46ab991-4cb2-4c42-83bf-8db4db680cfa None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:46.036220 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-d46ab991-4cb2-4c42-83bf-8db4db680cfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:46.050245 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46ab991-4cb2-4c42-83bf-8db4db680cfa None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:46.056647 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928b901b-aa38-4d6a-a23b-ea64232272bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:46.057453 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928b901b-aa38-4d6a-a23b-ea64232272bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:46.057859 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928b901b-aa38-4d6a-a23b-ea64232272bf admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:46.058240 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928b901b-aa38-4d6a-a23b-ea64232272bf admin admin] PATH_INFO: `/v3/projects/cbfe6f72bde949d5a190688dbde4bc7d` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:46.059525 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928b901b-aa38-4d6a-a23b-ea64232272bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbfe6f72bde949d5a190688dbde4bc7d)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:46.063282 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928b901b-aa38-4d6a-a23b-ea64232272bf admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:46.118462 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-928b901b-aa38-4d6a-a23b-ea64232272bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbfe6f72bde949d5a190688dbde4bc7d', 'request_id': 'req-928b901b-aa38-4d6a-a23b-ea64232272bf'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:46.125748 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-928b901b-aa38-4d6a-a23b-ea64232272bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbfe6f72bde949d5a190688dbde4bc7d', 'request_id': 'req-928b901b-aa38-4d6a-a23b-ea64232272bf'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:46.132565 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-928b901b-aa38-4d6a-a23b-ea64232272bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbfe6f72bde949d5a190688dbde4bc7d', 'request_id': 'req-928b901b-aa38-4d6a-a23b-ea64232272bf'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:46.148846 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-928b901b-aa38-4d6a-a23b-ea64232272bf admin admin] The token cache is being invalidate because project cbfe6f72bde949d5a190688dbde4bc7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 13:47:46.149228 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-928b901b-aa38-4d6a-a23b-ea64232272bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:46.149885 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-928b901b-aa38-4d6a-a23b-ea64232272bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:46.151655 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1046/2100] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 13:47:46 2022] DELETE /identity/v3/projects/cbfe6f72bde949d5a190688dbde4bc7d => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 13:47:46.156152 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3afc24b-d9fe-4a53-ae01-72c9c687edf3 service neutron] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:46.166533 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3afc24b-d9fe-4a53-ae01-72c9c687edf3 service neutron] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:46.230624 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6c505a-f81c-47b3-9c96-86d2477266ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:46.231507 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6c505a-f81c-47b3-9c96-86d2477266ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:46.231938 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6c505a-f81c-47b3-9c96-86d2477266ab admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:46.232317 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6c505a-f81c-47b3-9c96-86d2477266ab admin admin] PATH_INFO: `/v3/users/50a617499a8b4dcf96b7d104958d33bb` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:46.233681 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6c505a-f81c-47b3-9c96-86d2477266ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=50a617499a8b4dcf96b7d104958d33bb)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:46.235854 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6c505a-f81c-47b3-9c96-86d2477266ab admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:46.295749 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-fe6c505a-f81c-47b3-9c96-86d2477266ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50a617499a8b4dcf96b7d104958d33bb'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:46.303871 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-fe6c505a-f81c-47b3-9c96-86d2477266ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50a617499a8b4dcf96b7d104958d33bb'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:46.311858 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-fe6c505a-f81c-47b3-9c96-86d2477266ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50a617499a8b4dcf96b7d104958d33bb'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:46.312622 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-fe6c505a-f81c-47b3-9c96-86d2477266ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50a617499a8b4dcf96b7d104958d33bb'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:46.313228 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-fe6c505a-f81c-47b3-9c96-86d2477266ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50a617499a8b4dcf96b7d104958d33bb'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:46.341189 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1055/2101] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 13:47:46 2022] DELETE /identity/v3/users/50a617499a8b4dcf96b7d104958d33bb => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:47:46.946383 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928b901b-aa38-4d6a-a23b-ea64232272bf admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:46.947862 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-928b901b-aa38-4d6a-a23b-ea64232272bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:46.968249 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928b901b-aa38-4d6a-a23b-ea64232272bf admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:46.999421 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:47:47.030990 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4535c3f8-f6bc-4885-9bdc-85ea645f2f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:47.031648 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4535c3f8-f6bc-4885-9bdc-85ea645f2f35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:47.031935 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4535c3f8-f6bc-4885-9bdc-85ea645f2f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:47.032212 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4535c3f8-f6bc-4885-9bdc-85ea645f2f35 admin admin] PATH_INFO: `/v3/projects/3ffa008eaa95413e87f747a754a48606` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:47.033397 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4535c3f8-f6bc-4885-9bdc-85ea645f2f35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ffa008eaa95413e87f747a754a48606)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:47.035425 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4535c3f8-f6bc-4885-9bdc-85ea645f2f35 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:47.087836 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-4535c3f8-f6bc-4885-9bdc-85ea645f2f35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ffa008eaa95413e87f747a754a48606', 'request_id': 'req-4535c3f8-f6bc-4885-9bdc-85ea645f2f35'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:47.093981 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-4535c3f8-f6bc-4885-9bdc-85ea645f2f35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ffa008eaa95413e87f747a754a48606', 'request_id': 'req-4535c3f8-f6bc-4885-9bdc-85ea645f2f35'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:47.099392 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-4535c3f8-f6bc-4885-9bdc-85ea645f2f35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ffa008eaa95413e87f747a754a48606', 'request_id': 'req-4535c3f8-f6bc-4885-9bdc-85ea645f2f35'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:47.111282 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-4535c3f8-f6bc-4885-9bdc-85ea645f2f35 admin admin] The token cache is being invalidate because project 3ffa008eaa95413e87f747a754a48606 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 13:47:47.111571 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-4535c3f8-f6bc-4885-9bdc-85ea645f2f35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:47.112183 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-4535c3f8-f6bc-4885-9bdc-85ea645f2f35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:47.113717 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1047/2102] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 13:47:46 2022] DELETE /identity/v3/projects/3ffa008eaa95413e87f747a754a48606 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:47:47.118632 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6245c45f-a343-46e6-abc1-509c35cf9520 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:47.118907 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6245c45f-a343-46e6-abc1-509c35cf9520 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:47.119181 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6245c45f-a343-46e6-abc1-509c35cf9520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:47.139229 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-6245c45f-a343-46e6-abc1-509c35cf9520 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:47:47.140414 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-6245c45f-a343-46e6-abc1-509c35cf9520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:47.197608 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1056/2103] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:47:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:47:47.205703 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb609495-5b99-4042-8d37-1280e3b756e5 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:47.205987 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb609495-5b99-4042-8d37-1280e3b756e5 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:47.206261 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb609495-5b99-4042-8d37-1280e3b756e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:47.225793 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-eb609495-5b99-4042-8d37-1280e3b756e5 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:47:47.226985 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-eb609495-5b99-4042-8d37-1280e3b756e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:47.234794 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1048/2104] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:47:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:47:47.238880 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6245c45f-a343-46e6-abc1-509c35cf9520 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:47.249314 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6245c45f-a343-46e6-abc1-509c35cf9520 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:47.253904 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dd2ec4-3b36-44e5-9dfc-63fe5323147e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:47.254707 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dd2ec4-3b36-44e5-9dfc-63fe5323147e admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:47.254823 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dd2ec4-3b36-44e5-9dfc-63fe5323147e admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:47.255052 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dd2ec4-3b36-44e5-9dfc-63fe5323147e admin admin] PATH_INFO: `/v3/domains` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:47.255748 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dd2ec4-3b36-44e5-9dfc-63fe5323147e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:47:47.256059 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dd2ec4-3b36-44e5-9dfc-63fe5323147e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:47.257740 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dd2ec4-3b36-44e5-9dfc-63fe5323147e admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:47.259465 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:47:47.272160 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1057/2105] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Jul 26 13:47:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:47:47.276383 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb609495-5b99-4042-8d37-1280e3b756e5 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:47.286356 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb609495-5b99-4042-8d37-1280e3b756e5 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:47.290961 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cba9f4e-d102-4993-a49e-7ee58d16c8c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:47.291565 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cba9f4e-d102-4993-a49e-7ee58d16c8c8 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:47.291842 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cba9f4e-d102-4993-a49e-7ee58d16c8c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:47.292114 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cba9f4e-d102-4993-a49e-7ee58d16c8c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:47.292778 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cba9f4e-d102-4993-a49e-7ee58d16c8c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:47.294729 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cba9f4e-d102-4993-a49e-7ee58d16c8c8 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:47.321936 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1049/2106] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 13:47:47 2022] POST /identity/v3/projects => generated 381 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 13:47:47.326830 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dd2ec4-3b36-44e5-9dfc-63fe5323147e admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:47.337781 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dd2ec4-3b36-44e5-9dfc-63fe5323147e admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:47.397829 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d31515-35bf-4f88-8202-d768e106223d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:47.398467 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d31515-35bf-4f88-8202-d768e106223d admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:47.398755 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d31515-35bf-4f88-8202-d768e106223d admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:47.399033 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d31515-35bf-4f88-8202-d768e106223d admin admin] PATH_INFO: `/v3/users` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:47.399701 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d31515-35bf-4f88-8202-d768e106223d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:47.401644 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d31515-35bf-4f88-8202-d768e106223d admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:47.429284 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1058/2107] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 13:47:47 2022] POST /identity/v3/users => generated 291 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 13:47:47.430645 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cba9f4e-d102-4993-a49e-7ee58d16c8c8 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:47.443202 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cba9f4e-d102-4993-a49e-7ee58d16c8c8 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:47.448884 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a59cd4-5f27-430f-80f0-be4e64b104ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:47.449461 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a59cd4-5f27-430f-80f0-be4e64b104ac admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:47.450263 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a59cd4-5f27-430f-80f0-be4e64b104ac admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:47.450553 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a59cd4-5f27-430f-80f0-be4e64b104ac admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:47.451338 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a59cd4-5f27-430f-80f0-be4e64b104ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:47.452940 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a59cd4-5f27-430f-80f0-be4e64b104ac admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:47.464356 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1050/2108] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:47:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 13:47:47.468303 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d31515-35bf-4f88-8202-d768e106223d admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:47.479327 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d31515-35bf-4f88-8202-d768e106223d admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:47.485338 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a806739-f554-4feb-acae-e778a3858b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:47.485992 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a806739-f554-4feb-acae-e778a3858b53 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:47.486281 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a806739-f554-4feb-acae-e778a3858b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:47.486568 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a806739-f554-4feb-acae-e778a3858b53 admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:47.487261 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a806739-f554-4feb-acae-e778a3858b53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:47.488844 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a806739-f554-4feb-acae-e778a3858b53 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:47.501152 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1059/2109] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:47:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:47:47.505166 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a59cd4-5f27-430f-80f0-be4e64b104ac admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:47.515434 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a59cd4-5f27-430f-80f0-be4e64b104ac admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:47.520050 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19a3576-e014-45b8-9998-a4197575e995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:47.520704 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19a3576-e014-45b8-9998-a4197575e995 admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:47.520993 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19a3576-e014-45b8-9998-a4197575e995 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:47.521306 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19a3576-e014-45b8-9998-a4197575e995 admin admin] PATH_INFO: `/v3/projects/224834e5a44c44baaae417567bde46da/users/0479dbd800bd4f378d0428a0e12cb5e0/roles/759f389b221049c4b01faf3b984cd8d7` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:47.536953 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19a3576-e014-45b8-9998-a4197575e995 admin admin] RBAC: Authorizing `identity:create_grant(project_id=224834e5a44c44baaae417567bde46da, user_id=0479dbd800bd4f378d0428a0e12cb5e0, role_id=759f389b221049c4b01faf3b984cd8d7)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:47.539619 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19a3576-e014-45b8-9998-a4197575e995 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:47.553106 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1051/2110] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 13:47:47 2022] PUT /identity/v3/projects/224834e5a44c44baaae417567bde46da/users/0479dbd800bd4f378d0428a0e12cb5e0/roles/759f389b221049c4b01faf3b984cd8d7 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:47:47.563459 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e3572b-521a-42b1-849b-b55077ef6b80 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:47.563749 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e3572b-521a-42b1-849b-b55077ef6b80 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:47.564028 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e3572b-521a-42b1-849b-b55077ef6b80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:47.591174 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-d7e3572b-521a-42b1-849b-b55077ef6b80 None None] MFA Rules not processed for user `0479dbd800bd4f378d0428a0e12cb5e0`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:47:47.592873 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-d7e3572b-521a-42b1-849b-b55077ef6b80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:47.655074 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1060/2111] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:47:47 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:47:47.659704 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19a3576-e014-45b8-9998-a4197575e995 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:47.727624 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:47:47.732795 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19a3576-e014-45b8-9998-a4197575e995 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:47.739128 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f84aa1-fd37-42a1-bf70-5b1bf8bab1fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:47.739926 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f84aa1-fd37-42a1-bf70-5b1bf8bab1fc admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:47.740380 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f84aa1-fd37-42a1-bf70-5b1bf8bab1fc admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:47.740653 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f84aa1-fd37-42a1-bf70-5b1bf8bab1fc admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:47.741314 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f84aa1-fd37-42a1-bf70-5b1bf8bab1fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:47.743400 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f84aa1-fd37-42a1-bf70-5b1bf8bab1fc admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:47.779638 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1052/2112] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 13:47:47 2022] POST /identity/v3/projects => generated 379 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:47:47.783990 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e3572b-521a-42b1-849b-b55077ef6b80 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:47.795479 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e3572b-521a-42b1-849b-b55077ef6b80 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:47.851617 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7698f7a-d796-4d09-b7fb-c6e0ed7c91f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:47.852230 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7698f7a-d796-4d09-b7fb-c6e0ed7c91f2 admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:47.852509 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7698f7a-d796-4d09-b7fb-c6e0ed7c91f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:47.852785 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7698f7a-d796-4d09-b7fb-c6e0ed7c91f2 admin admin] PATH_INFO: `/v3/users` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:47.853440 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7698f7a-d796-4d09-b7fb-c6e0ed7c91f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:47.855646 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7698f7a-d796-4d09-b7fb-c6e0ed7c91f2 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:47.864706 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:47:47.880981 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1061/2113] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 13:47:47 2022] POST /identity/v3/users => generated 296 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 13:47:47.882583 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f84aa1-fd37-42a1-bf70-5b1bf8bab1fc admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:47.893754 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f84aa1-fd37-42a1-bf70-5b1bf8bab1fc admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:47.898474 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6e64dd-a90b-44ec-8f4b-eba85dd918af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:47.899089 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6e64dd-a90b-44ec-8f4b-eba85dd918af admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:47.899391 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6e64dd-a90b-44ec-8f4b-eba85dd918af admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:47.899666 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6e64dd-a90b-44ec-8f4b-eba85dd918af admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:47.900354 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6e64dd-a90b-44ec-8f4b-eba85dd918af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:47.902032 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6e64dd-a90b-44ec-8f4b-eba85dd918af admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:47.913170 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1053/2114] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:47:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:47:47.917505 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7698f7a-d796-4d09-b7fb-c6e0ed7c91f2 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:47.928672 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7698f7a-d796-4d09-b7fb-c6e0ed7c91f2 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:47.933737 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea0c4b1-dbcf-45c5-b895-abf0f8d231ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:47.934428 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea0c4b1-dbcf-45c5-b895-abf0f8d231ae admin admin] REQUEST_METHOD: `PUT` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:47.934696 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea0c4b1-dbcf-45c5-b895-abf0f8d231ae admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:47.934971 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea0c4b1-dbcf-45c5-b895-abf0f8d231ae admin admin] PATH_INFO: `/v3/projects/313d3389da7344c295b662c57b32adc1/users/147615d69a8b493ba8595a63515c48dd/roles/6e0201a9a6484f7d8b112b1eafff70bd` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:47.951587 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea0c4b1-dbcf-45c5-b895-abf0f8d231ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=313d3389da7344c295b662c57b32adc1, user_id=147615d69a8b493ba8595a63515c48dd, role_id=6e0201a9a6484f7d8b112b1eafff70bd)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:47.954329 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea0c4b1-dbcf-45c5-b895-abf0f8d231ae admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:47.967877 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1062/2115] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 13:47:47 2022] PUT /identity/v3/projects/313d3389da7344c295b662c57b32adc1/users/147615d69a8b493ba8595a63515c48dd/roles/6e0201a9a6484f7d8b112b1eafff70bd => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 13:47:47.972078 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6e64dd-a90b-44ec-8f4b-eba85dd918af admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:47.982616 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6e64dd-a90b-44ec-8f4b-eba85dd918af admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:48.042971 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd818bf5-f371-47b2-952d-1a914db1c527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:48.043607 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd818bf5-f371-47b2-952d-1a914db1c527 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:48.043891 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd818bf5-f371-47b2-952d-1a914db1c527 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:48.044168 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd818bf5-f371-47b2-952d-1a914db1c527 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:48.045402 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd818bf5-f371-47b2-952d-1a914db1c527 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:48.047002 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd818bf5-f371-47b2-952d-1a914db1c527 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:48.057226 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1054/2116] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:47:47 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:47:48.061856 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea0c4b1-dbcf-45c5-b895-abf0f8d231ae admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:48.072475 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea0c4b1-dbcf-45c5-b895-abf0f8d231ae admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:48.078109 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99dd42a-0f20-4f65-af81-378bde8eaeac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:48.078832 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99dd42a-0f20-4f65-af81-378bde8eaeac admin admin] REQUEST_METHOD: `PUT` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:48.079121 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99dd42a-0f20-4f65-af81-378bde8eaeac admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:48.079405 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99dd42a-0f20-4f65-af81-378bde8eaeac admin admin] PATH_INFO: `/v3/projects/313d3389da7344c295b662c57b32adc1/users/147615d69a8b493ba8595a63515c48dd/roles/6e0201a9a6484f7d8b112b1eafff70bd` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:48.081393 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99dd42a-0f20-4f65-af81-378bde8eaeac admin admin] RBAC: Authorizing `identity:create_grant(project_id=313d3389da7344c295b662c57b32adc1, user_id=147615d69a8b493ba8595a63515c48dd, role_id=6e0201a9a6484f7d8b112b1eafff70bd)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:48.084162 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99dd42a-0f20-4f65-af81-378bde8eaeac admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:48.095218 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1063/2117] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 13:47:48 2022] PUT /identity/v3/projects/313d3389da7344c295b662c57b32adc1/users/147615d69a8b493ba8595a63515c48dd/roles/6e0201a9a6484f7d8b112b1eafff70bd => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:47:48.106295 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd40bbbb-718f-4e2e-852d-5a4a9b615866 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:48.106584 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd40bbbb-718f-4e2e-852d-5a4a9b615866 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:48.106860 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd40bbbb-718f-4e2e-852d-5a4a9b615866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:48.133775 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-dd40bbbb-718f-4e2e-852d-5a4a9b615866 None None] MFA Rules not processed for user `147615d69a8b493ba8595a63515c48dd`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:47:48.134971 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-dd40bbbb-718f-4e2e-852d-5a4a9b615866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:48.203251 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1055/2118] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:47:48 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:47:48.215348 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99dd42a-0f20-4f65-af81-378bde8eaeac admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:48.217691 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-c99dd42a-0f20-4f65-af81-378bde8eaeac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:48.274257 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99dd42a-0f20-4f65-af81-378bde8eaeac admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:48.277883 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e811a3a7-21c5-43e9-a280-db23fa5cd22d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:48.278542 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e811a3a7-21c5-43e9-a280-db23fa5cd22d service cinder] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:48.278852 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e811a3a7-21c5-43e9-a280-db23fa5cd22d service cinder] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:48.279974 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e811a3a7-21c5-43e9-a280-db23fa5cd22d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:48.288820 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e811a3a7-21c5-43e9-a280-db23fa5cd22d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:48.290266 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e811a3a7-21c5-43e9-a280-db23fa5cd22d service cinder] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:48.299654 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1064/2119] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:47:48 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:47:48.636056 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd40bbbb-718f-4e2e-852d-5a4a9b615866 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:48.645836 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd40bbbb-718f-4e2e-852d-5a4a9b615866 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:48.651299 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c329ef-0cbe-4f59-88e4-7c8a2dff7b8e tempest-VolumeQuotaClassesTest-105318182 tempest-VolumeQuotaClassesTest-105318182-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '147615d69a8b493ba8595a63515c48dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '313d3389da7344c295b662c57b32adc1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:48.651923 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c329ef-0cbe-4f59-88e4-7c8a2dff7b8e tempest-VolumeQuotaClassesTest-105318182 tempest-VolumeQuotaClassesTest-105318182-project-admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:48.652212 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c329ef-0cbe-4f59-88e4-7c8a2dff7b8e tempest-VolumeQuotaClassesTest-105318182 tempest-VolumeQuotaClassesTest-105318182-project-admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:48.652487 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c329ef-0cbe-4f59-88e4-7c8a2dff7b8e tempest-VolumeQuotaClassesTest-105318182 tempest-VolumeQuotaClassesTest-105318182-project-admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:48.653186 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c329ef-0cbe-4f59-88e4-7c8a2dff7b8e tempest-VolumeQuotaClassesTest-105318182 tempest-VolumeQuotaClassesTest-105318182-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:47:48.653509 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c329ef-0cbe-4f59-88e4-7c8a2dff7b8e tempest-VolumeQuotaClassesTest-105318182 tempest-VolumeQuotaClassesTest-105318182-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:48.655985 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c329ef-0cbe-4f59-88e4-7c8a2dff7b8e tempest-VolumeQuotaClassesTest-105318182 tempest-VolumeQuotaClassesTest-105318182-project-admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:48.668401 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1056/2120] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Jul 26 13:47:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:47:48.674534 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e811a3a7-21c5-43e9-a280-db23fa5cd22d service cinder] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:48.684608 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e811a3a7-21c5-43e9-a280-db23fa5cd22d service cinder] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:48.689296 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a684a8f5-e0f1-4ccf-99f5-91b80e4000b4 tempest-VolumeQuotaClassesTest-105318182 tempest-VolumeQuotaClassesTest-105318182-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '147615d69a8b493ba8595a63515c48dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '313d3389da7344c295b662c57b32adc1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:48.690056 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a684a8f5-e0f1-4ccf-99f5-91b80e4000b4 tempest-VolumeQuotaClassesTest-105318182 tempest-VolumeQuotaClassesTest-105318182-project-admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:48.690415 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a684a8f5-e0f1-4ccf-99f5-91b80e4000b4 tempest-VolumeQuotaClassesTest-105318182 tempest-VolumeQuotaClassesTest-105318182-project-admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:48.690768 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a684a8f5-e0f1-4ccf-99f5-91b80e4000b4 tempest-VolumeQuotaClassesTest-105318182 tempest-VolumeQuotaClassesTest-105318182-project-admin] PATH_INFO: `/v3/projects` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:48.691497 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a684a8f5-e0f1-4ccf-99f5-91b80e4000b4 tempest-VolumeQuotaClassesTest-105318182 tempest-VolumeQuotaClassesTest-105318182-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:48.693463 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a684a8f5-e0f1-4ccf-99f5-91b80e4000b4 tempest-VolumeQuotaClassesTest-105318182 tempest-VolumeQuotaClassesTest-105318182-project-admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:48.720902 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1065/2121] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 13:47:48 2022] POST /identity/v3/projects => generated 355 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:47:48.724987 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c329ef-0cbe-4f59-88e4-7c8a2dff7b8e tempest-VolumeQuotaClassesTest-105318182 tempest-VolumeQuotaClassesTest-105318182-project-admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:48.735527 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c329ef-0cbe-4f59-88e4-7c8a2dff7b8e tempest-VolumeQuotaClassesTest-105318182 tempest-VolumeQuotaClassesTest-105318182-project-admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:48.773602 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:47:48.796266 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09f9881-958e-4711-b6eb-0ffacf7a520f tempest-VolumeQuotaClassesTest-105318182 tempest-VolumeQuotaClassesTest-105318182-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '147615d69a8b493ba8595a63515c48dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '313d3389da7344c295b662c57b32adc1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:48.796965 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09f9881-958e-4711-b6eb-0ffacf7a520f tempest-VolumeQuotaClassesTest-105318182 tempest-VolumeQuotaClassesTest-105318182-project-admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:48.797251 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09f9881-958e-4711-b6eb-0ffacf7a520f tempest-VolumeQuotaClassesTest-105318182 tempest-VolumeQuotaClassesTest-105318182-project-admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:48.797587 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09f9881-958e-4711-b6eb-0ffacf7a520f tempest-VolumeQuotaClassesTest-105318182 tempest-VolumeQuotaClassesTest-105318182-project-admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:48.798299 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09f9881-958e-4711-b6eb-0ffacf7a520f tempest-VolumeQuotaClassesTest-105318182 tempest-VolumeQuotaClassesTest-105318182-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:47:48.798644 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09f9881-958e-4711-b6eb-0ffacf7a520f tempest-VolumeQuotaClassesTest-105318182 tempest-VolumeQuotaClassesTest-105318182-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:48.800380 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09f9881-958e-4711-b6eb-0ffacf7a520f tempest-VolumeQuotaClassesTest-105318182 tempest-VolumeQuotaClassesTest-105318182-project-admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:48.817314 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1057/2122] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Jul 26 13:47:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:47:48.855751 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a684a8f5-e0f1-4ccf-99f5-91b80e4000b4 tempest-VolumeQuotaClassesTest-105318182 tempest-VolumeQuotaClassesTest-105318182-project-admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:48.866839 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a684a8f5-e0f1-4ccf-99f5-91b80e4000b4 tempest-VolumeQuotaClassesTest-105318182 tempest-VolumeQuotaClassesTest-105318182-project-admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:48.874175 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc48388-e9fa-4533-9f32-0fe0b3e9dc9f tempest-VolumeQuotaClassesTest-105318182 tempest-VolumeQuotaClassesTest-105318182-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '147615d69a8b493ba8595a63515c48dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '313d3389da7344c295b662c57b32adc1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:48.874835 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc48388-e9fa-4533-9f32-0fe0b3e9dc9f tempest-VolumeQuotaClassesTest-105318182 tempest-VolumeQuotaClassesTest-105318182-project-admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:48.875118 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc48388-e9fa-4533-9f32-0fe0b3e9dc9f tempest-VolumeQuotaClassesTest-105318182 tempest-VolumeQuotaClassesTest-105318182-project-admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:48.875392 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc48388-e9fa-4533-9f32-0fe0b3e9dc9f tempest-VolumeQuotaClassesTest-105318182 tempest-VolumeQuotaClassesTest-105318182-project-admin] PATH_INFO: `/v3/projects/cda31f12d45747e682e433f2a67dd62d` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:48.876536 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc48388-e9fa-4533-9f32-0fe0b3e9dc9f tempest-VolumeQuotaClassesTest-105318182 tempest-VolumeQuotaClassesTest-105318182-project-admin] RBAC: Authorizing `identity:delete_project(project_id=cda31f12d45747e682e433f2a67dd62d)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:48.878459 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc48388-e9fa-4533-9f32-0fe0b3e9dc9f tempest-VolumeQuotaClassesTest-105318182 tempest-VolumeQuotaClassesTest-105318182-project-admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:48.929148 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-9dc48388-e9fa-4533-9f32-0fe0b3e9dc9f tempest-VolumeQuotaClassesTest-105318182 tempest-VolumeQuotaClassesTest-105318182-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cda31f12d45747e682e433f2a67dd62d', 'request_id': 'req-9dc48388-e9fa-4533-9f32-0fe0b3e9dc9f'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:48.937970 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-9dc48388-e9fa-4533-9f32-0fe0b3e9dc9f tempest-VolumeQuotaClassesTest-105318182 tempest-VolumeQuotaClassesTest-105318182-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cda31f12d45747e682e433f2a67dd62d', 'request_id': 'req-9dc48388-e9fa-4533-9f32-0fe0b3e9dc9f'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:48.945755 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-9dc48388-e9fa-4533-9f32-0fe0b3e9dc9f tempest-VolumeQuotaClassesTest-105318182 tempest-VolumeQuotaClassesTest-105318182-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cda31f12d45747e682e433f2a67dd62d', 'request_id': 'req-9dc48388-e9fa-4533-9f32-0fe0b3e9dc9f'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:48.959354 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-9dc48388-e9fa-4533-9f32-0fe0b3e9dc9f tempest-VolumeQuotaClassesTest-105318182 tempest-VolumeQuotaClassesTest-105318182-project-admin] The token cache is being invalidate because project cda31f12d45747e682e433f2a67dd62d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86933) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 13:47:48.959647 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-9dc48388-e9fa-4533-9f32-0fe0b3e9dc9f tempest-VolumeQuotaClassesTest-105318182 tempest-VolumeQuotaClassesTest-105318182-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:48.960321 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-9dc48388-e9fa-4533-9f32-0fe0b3e9dc9f tempest-VolumeQuotaClassesTest-105318182 tempest-VolumeQuotaClassesTest-105318182-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:48.962337 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1066/2123] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 13:47:48 2022] DELETE /identity/v3/projects/cda31f12d45747e682e433f2a67dd62d => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:47:49.198281 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09f9881-958e-4711-b6eb-0ffacf7a520f tempest-VolumeQuotaClassesTest-105318182 tempest-VolumeQuotaClassesTest-105318182-project-admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:49.200579 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-f09f9881-958e-4711-b6eb-0ffacf7a520f tempest-VolumeQuotaClassesTest-105318182 tempest-VolumeQuotaClassesTest-105318182-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:49.289676 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09f9881-958e-4711-b6eb-0ffacf7a520f tempest-VolumeQuotaClassesTest-105318182 tempest-VolumeQuotaClassesTest-105318182-project-admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:49.351636 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f297ef6a-07eb-4ed2-a09c-920f75f49a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:49.352283 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f297ef6a-07eb-4ed2-a09c-920f75f49a0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:49.352566 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f297ef6a-07eb-4ed2-a09c-920f75f49a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:49.352843 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f297ef6a-07eb-4ed2-a09c-920f75f49a0c admin admin] PATH_INFO: `/v3/users/0479dbd800bd4f378d0428a0e12cb5e0` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:49.354031 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f297ef6a-07eb-4ed2-a09c-920f75f49a0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=0479dbd800bd4f378d0428a0e12cb5e0)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:49.356075 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f297ef6a-07eb-4ed2-a09c-920f75f49a0c admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:49.403390 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-f297ef6a-07eb-4ed2-a09c-920f75f49a0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0479dbd800bd4f378d0428a0e12cb5e0'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:49.404992 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-f297ef6a-07eb-4ed2-a09c-920f75f49a0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0479dbd800bd4f378d0428a0e12cb5e0'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:49.411878 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-f297ef6a-07eb-4ed2-a09c-920f75f49a0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0479dbd800bd4f378d0428a0e12cb5e0'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:49.412508 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-f297ef6a-07eb-4ed2-a09c-920f75f49a0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0479dbd800bd4f378d0428a0e12cb5e0'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:49.424231 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-f297ef6a-07eb-4ed2-a09c-920f75f49a0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0479dbd800bd4f378d0428a0e12cb5e0'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:49.432971 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:47:49.437836 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1058/2124] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 13:47:49 2022] DELETE /identity/v3/users/0479dbd800bd4f378d0428a0e12cb5e0 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:47:49.442613 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f9a3d0-4611-409f-b80a-39e45ed82b62 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:49.443221 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f9a3d0-4611-409f-b80a-39e45ed82b62 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:49.443343 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f9a3d0-4611-409f-b80a-39e45ed82b62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:49.462519 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-34f9a3d0-4611-409f-b80a-39e45ed82b62 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:47:49.463746 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-34f9a3d0-4611-409f-b80a-39e45ed82b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:49.475831 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:INSERT=6,keystone:SELECT=87,keystone:DELETE=7 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:47:49.522557 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1067/2125] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:47:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:47:49.533094 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f297ef6a-07eb-4ed2-a09c-920f75f49a0c admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:49.534528 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-f297ef6a-07eb-4ed2-a09c-920f75f49a0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:49.593817 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f297ef6a-07eb-4ed2-a09c-920f75f49a0c admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:49.597251 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ac4958-4f0e-4e08-ae50-1eed6af434c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:49.597894 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ac4958-4f0e-4e08-ae50-1eed6af434c2 service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:49.598183 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ac4958-4f0e-4e08-ae50-1eed6af434c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:49.598455 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ac4958-4f0e-4e08-ae50-1eed6af434c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:49.608577 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ac4958-4f0e-4e08-ae50-1eed6af434c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:49.610386 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ac4958-4f0e-4e08-ae50-1eed6af434c2 service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:49.619730 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1059/2126] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:47:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:47:49.902999 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f9a3d0-4611-409f-b80a-39e45ed82b62 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:49.905287 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-34f9a3d0-4611-409f-b80a-39e45ed82b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:49.923943 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f9a3d0-4611-409f-b80a-39e45ed82b62 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:49.929379 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db41a6f4-e800-4682-9ccf-9d81e307700a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:49.930244 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db41a6f4-e800-4682-9ccf-9d81e307700a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:49.930683 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db41a6f4-e800-4682-9ccf-9d81e307700a admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:49.931111 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db41a6f4-e800-4682-9ccf-9d81e307700a admin admin] PATH_INFO: `/v3/projects/224834e5a44c44baaae417567bde46da` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:49.932581 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db41a6f4-e800-4682-9ccf-9d81e307700a admin admin] RBAC: Authorizing `identity:delete_project(project_id=224834e5a44c44baaae417567bde46da)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:49.934782 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db41a6f4-e800-4682-9ccf-9d81e307700a admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:49.995353 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-db41a6f4-e800-4682-9ccf-9d81e307700a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '224834e5a44c44baaae417567bde46da', 'request_id': 'req-db41a6f4-e800-4682-9ccf-9d81e307700a'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:50.004133 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-db41a6f4-e800-4682-9ccf-9d81e307700a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '224834e5a44c44baaae417567bde46da', 'request_id': 'req-db41a6f4-e800-4682-9ccf-9d81e307700a'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:50.009833 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-db41a6f4-e800-4682-9ccf-9d81e307700a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '224834e5a44c44baaae417567bde46da', 'request_id': 'req-db41a6f4-e800-4682-9ccf-9d81e307700a'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:50.022062 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-db41a6f4-e800-4682-9ccf-9d81e307700a admin admin] The token cache is being invalidate because project 224834e5a44c44baaae417567bde46da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86933) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 13:47:50.022444 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-db41a6f4-e800-4682-9ccf-9d81e307700a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:50.026138 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-db41a6f4-e800-4682-9ccf-9d81e307700a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:50.027715 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1068/2127] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 13:47:49 2022] DELETE /identity/v3/projects/224834e5a44c44baaae417567bde46da => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:47:50.032985 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ac4958-4f0e-4e08-ae50-1eed6af434c2 service neutron] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:50.044178 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ac4958-4f0e-4e08-ae50-1eed6af434c2 service neutron] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:50.108681 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdad0306-5d0b-4121-8770-5336e0bfa455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:50.109352 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdad0306-5d0b-4121-8770-5336e0bfa455 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:50.109667 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdad0306-5d0b-4121-8770-5336e0bfa455 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:50.109944 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdad0306-5d0b-4121-8770-5336e0bfa455 admin admin] PATH_INFO: `/v3/users/147615d69a8b493ba8595a63515c48dd` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:50.111104 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdad0306-5d0b-4121-8770-5336e0bfa455 admin admin] RBAC: Authorizing `identity:delete_user(user_id=147615d69a8b493ba8595a63515c48dd)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:50.114197 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdad0306-5d0b-4121-8770-5336e0bfa455 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:50.170618 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-fdad0306-5d0b-4121-8770-5336e0bfa455 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '147615d69a8b493ba8595a63515c48dd'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:50.171290 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-fdad0306-5d0b-4121-8770-5336e0bfa455 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '147615d69a8b493ba8595a63515c48dd'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:50.178920 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-fdad0306-5d0b-4121-8770-5336e0bfa455 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '147615d69a8b493ba8595a63515c48dd'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:50.179706 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-fdad0306-5d0b-4121-8770-5336e0bfa455 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '147615d69a8b493ba8595a63515c48dd'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:50.193165 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-fdad0306-5d0b-4121-8770-5336e0bfa455 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '147615d69a8b493ba8595a63515c48dd'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:50.206416 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1060/2128] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 13:47:50 2022] DELETE /identity/v3/users/147615d69a8b493ba8595a63515c48dd => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:47:50.507326 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db41a6f4-e800-4682-9ccf-9d81e307700a admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:50.509875 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-db41a6f4-e800-4682-9ccf-9d81e307700a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:50.524356 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db41a6f4-e800-4682-9ccf-9d81e307700a admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:50.585502 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5e48bd-2f1c-4b7d-b21b-c72531556b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:50.586195 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5e48bd-2f1c-4b7d-b21b-c72531556b58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:50.586533 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5e48bd-2f1c-4b7d-b21b-c72531556b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:50.586782 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5e48bd-2f1c-4b7d-b21b-c72531556b58 admin admin] PATH_INFO: `/v3/projects/313d3389da7344c295b662c57b32adc1` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:50.588040 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5e48bd-2f1c-4b7d-b21b-c72531556b58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=313d3389da7344c295b662c57b32adc1)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:50.590010 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5e48bd-2f1c-4b7d-b21b-c72531556b58 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:50.634501 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:47:50.638717 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-fc5e48bd-2f1c-4b7d-b21b-c72531556b58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '313d3389da7344c295b662c57b32adc1', 'request_id': 'req-fc5e48bd-2f1c-4b7d-b21b-c72531556b58'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:50.646986 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-fc5e48bd-2f1c-4b7d-b21b-c72531556b58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '313d3389da7344c295b662c57b32adc1', 'request_id': 'req-fc5e48bd-2f1c-4b7d-b21b-c72531556b58'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:50.653421 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-fc5e48bd-2f1c-4b7d-b21b-c72531556b58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '313d3389da7344c295b662c57b32adc1', 'request_id': 'req-fc5e48bd-2f1c-4b7d-b21b-c72531556b58'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:50.666777 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-fc5e48bd-2f1c-4b7d-b21b-c72531556b58 admin admin] The token cache is being invalidate because project 313d3389da7344c295b662c57b32adc1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86933) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 13:47:50.667100 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-fc5e48bd-2f1c-4b7d-b21b-c72531556b58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:50.667772 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-fc5e48bd-2f1c-4b7d-b21b-c72531556b58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:50.669539 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1069/2129] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 13:47:50 2022] DELETE /identity/v3/projects/313d3389da7344c295b662c57b32adc1 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:47:50.675093 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d05960-bbab-492a-a5e1-516bd1b54a04 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:50.675488 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d05960-bbab-492a-a5e1-516bd1b54a04 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:50.675917 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d05960-bbab-492a-a5e1-516bd1b54a04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:50.700790 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-34d05960-bbab-492a-a5e1-516bd1b54a04 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:47:50.702151 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-34d05960-bbab-492a-a5e1-516bd1b54a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:50.721966 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:47:50.772569 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1061/2130] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:47:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:47:50.781006 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fc232b-5b3a-4768-bb85-e624c2969d29 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:50.781378 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fc232b-5b3a-4768-bb85-e624c2969d29 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:50.781607 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fc232b-5b3a-4768-bb85-e624c2969d29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:50.804345 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-b5fc232b-5b3a-4768-bb85-e624c2969d29 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:47:50.805730 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-b5fc232b-5b3a-4768-bb85-e624c2969d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:50.814456 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1070/2131] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:47:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:47:50.819138 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d05960-bbab-492a-a5e1-516bd1b54a04 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:50.831708 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d05960-bbab-492a-a5e1-516bd1b54a04 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:50.835583 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08b10ab-737d-463c-bbe7-52a0483d8b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:50.836229 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08b10ab-737d-463c-bbe7-52a0483d8b51 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:50.836501 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08b10ab-737d-463c-bbe7-52a0483d8b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:50.836841 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08b10ab-737d-463c-bbe7-52a0483d8b51 admin admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:50.837506 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08b10ab-737d-463c-bbe7-52a0483d8b51 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:47:50.837821 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08b10ab-737d-463c-bbe7-52a0483d8b51 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:50.839417 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08b10ab-737d-463c-bbe7-52a0483d8b51 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:50.855622 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1062/2132] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Jul 26 13:47:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:47:50.856839 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fc232b-5b3a-4768-bb85-e624c2969d29 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:50.866717 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fc232b-5b3a-4768-bb85-e624c2969d29 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:50.878782 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae544b34-1ce9-4d8d-862c-61f381fb420d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:50.879686 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae544b34-1ce9-4d8d-862c-61f381fb420d admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:50.879866 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae544b34-1ce9-4d8d-862c-61f381fb420d admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:50.880143 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae544b34-1ce9-4d8d-862c-61f381fb420d admin admin] PATH_INFO: `/v3/projects` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:50.880953 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae544b34-1ce9-4d8d-862c-61f381fb420d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:50.882780 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae544b34-1ce9-4d8d-862c-61f381fb420d admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:50.923518 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1071/2133] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 13:47:50 2022] POST /identity/v3/projects => generated 385 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:47:50.927913 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08b10ab-737d-463c-bbe7-52a0483d8b51 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:50.940135 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08b10ab-737d-463c-bbe7-52a0483d8b51 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:51.016100 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c39722-46a9-4937-809a-0daa67b37bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:51.016825 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c39722-46a9-4937-809a-0daa67b37bc5 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:51.017168 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c39722-46a9-4937-809a-0daa67b37bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:51.017537 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c39722-46a9-4937-809a-0daa67b37bc5 admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:51.018224 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c39722-46a9-4937-809a-0daa67b37bc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:51.020123 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c39722-46a9-4937-809a-0daa67b37bc5 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:51.052282 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1063/2134] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 13:47:50 2022] POST /identity/v3/users => generated 293 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 13:47:51.053624 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae544b34-1ce9-4d8d-862c-61f381fb420d admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:51.063906 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae544b34-1ce9-4d8d-862c-61f381fb420d admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:51.068492 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9ff322-6399-42b7-bf87-35faeafbd3ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:51.069104 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9ff322-6399-42b7-bf87-35faeafbd3ce admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:51.069386 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9ff322-6399-42b7-bf87-35faeafbd3ce admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:51.069684 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9ff322-6399-42b7-bf87-35faeafbd3ce admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:51.070383 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9ff322-6399-42b7-bf87-35faeafbd3ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:51.071958 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9ff322-6399-42b7-bf87-35faeafbd3ce admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:51.082219 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1072/2135] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:47:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:47:51.086312 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c39722-46a9-4937-809a-0daa67b37bc5 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:51.096765 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c39722-46a9-4937-809a-0daa67b37bc5 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:51.101193 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722492b6-cd6a-4996-ae21-36aa991df126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:51.101826 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722492b6-cd6a-4996-ae21-36aa991df126 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:51.102110 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722492b6-cd6a-4996-ae21-36aa991df126 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:51.102379 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722492b6-cd6a-4996-ae21-36aa991df126 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:51.103059 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722492b6-cd6a-4996-ae21-36aa991df126 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:51.104626 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722492b6-cd6a-4996-ae21-36aa991df126 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:51.115346 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1064/2136] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:47:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:47:51.119168 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9ff322-6399-42b7-bf87-35faeafbd3ce admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:51.129005 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9ff322-6399-42b7-bf87-35faeafbd3ce admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:51.133678 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c54398-2877-4f06-9878-15ea9824c89a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:51.134331 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c54398-2877-4f06-9878-15ea9824c89a admin admin] REQUEST_METHOD: `PUT` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:51.134614 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c54398-2877-4f06-9878-15ea9824c89a admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:51.134890 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c54398-2877-4f06-9878-15ea9824c89a admin admin] PATH_INFO: `/v3/projects/1166971ce3a546e2af32d938e70af473/users/b0ab918b2f1a481fb7715491ea863798/roles/759f389b221049c4b01faf3b984cd8d7` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:51.149384 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c54398-2877-4f06-9878-15ea9824c89a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1166971ce3a546e2af32d938e70af473, user_id=b0ab918b2f1a481fb7715491ea863798, role_id=759f389b221049c4b01faf3b984cd8d7)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:51.152052 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c54398-2877-4f06-9878-15ea9824c89a admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:51.165727 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1073/2137] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 13:47:51 2022] PUT /identity/v3/projects/1166971ce3a546e2af32d938e70af473/users/b0ab918b2f1a481fb7715491ea863798/roles/759f389b221049c4b01faf3b984cd8d7 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:47:51.177491 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b97dd10-fcb4-4430-b4aa-857d384d2e96 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:51.177792 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b97dd10-fcb4-4430-b4aa-857d384d2e96 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:51.178069 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b97dd10-fcb4-4430-b4aa-857d384d2e96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:51.207010 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-0b97dd10-fcb4-4430-b4aa-857d384d2e96 None None] MFA Rules not processed for user `b0ab918b2f1a481fb7715491ea863798`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:47:51.208442 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-0b97dd10-fcb4-4430-b4aa-857d384d2e96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:51.273769 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1065/2138] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:47:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:47:51.285593 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c54398-2877-4f06-9878-15ea9824c89a admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:51.287304 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-c7c54398-2877-4f06-9878-15ea9824c89a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:51.343580 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c54398-2877-4f06-9878-15ea9824c89a admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:51.347300 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb717bc5-f04e-4c68-be66-ba1779e33a8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:51.347855 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb717bc5-f04e-4c68-be66-ba1779e33a8c service cinder] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:51.348135 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb717bc5-f04e-4c68-be66-ba1779e33a8c service cinder] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:51.348412 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb717bc5-f04e-4c68-be66-ba1779e33a8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:51.357685 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb717bc5-f04e-4c68-be66-ba1779e33a8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:51.359152 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb717bc5-f04e-4c68-be66-ba1779e33a8c service cinder] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:51.368050 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1074/2139] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:47:51 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:47:51.386976 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b97dd10-fcb4-4430-b4aa-857d384d2e96 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:51.409604 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:47:51.455243 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b97dd10-fcb4-4430-b4aa-857d384d2e96 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:51.460710 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054f341c-f914-4dac-9d68-a2254803dce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:51.461380 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054f341c-f914-4dac-9d68-a2254803dce2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:51.461725 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054f341c-f914-4dac-9d68-a2254803dce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:51.462006 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054f341c-f914-4dac-9d68-a2254803dce2 admin admin] PATH_INFO: `/v3/users/b0ab918b2f1a481fb7715491ea863798` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:51.463369 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054f341c-f914-4dac-9d68-a2254803dce2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0ab918b2f1a481fb7715491ea863798)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:51.465307 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054f341c-f914-4dac-9d68-a2254803dce2 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:51.512645 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-054f341c-f914-4dac-9d68-a2254803dce2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0ab918b2f1a481fb7715491ea863798'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:51.513384 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-054f341c-f914-4dac-9d68-a2254803dce2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0ab918b2f1a481fb7715491ea863798'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:51.520443 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-054f341c-f914-4dac-9d68-a2254803dce2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0ab918b2f1a481fb7715491ea863798'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:51.521111 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-054f341c-f914-4dac-9d68-a2254803dce2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0ab918b2f1a481fb7715491ea863798'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:51.533735 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-054f341c-f914-4dac-9d68-a2254803dce2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0ab918b2f1a481fb7715491ea863798'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:51.547340 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1066/2140] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 13:47:51 2022] DELETE /identity/v3/users/b0ab918b2f1a481fb7715491ea863798 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:47:51.551874 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b50ef5-56a9-434d-ba68-c148bf16b44e None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:51.552333 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b50ef5-56a9-434d-ba68-c148bf16b44e None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:51.552448 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b50ef5-56a9-434d-ba68-c148bf16b44e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:51.571465 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-74b50ef5-56a9-434d-ba68-c148bf16b44e None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:47:51.572737 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-74b50ef5-56a9-434d-ba68-c148bf16b44e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:51.634544 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1075/2141] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:47:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:47:51.644523 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054f341c-f914-4dac-9d68-a2254803dce2 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:51.645943 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-054f341c-f914-4dac-9d68-a2254803dce2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:51.703103 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054f341c-f914-4dac-9d68-a2254803dce2 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:51.706747 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3210f8-b1e5-469b-a599-a9dac2603c17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:51.707366 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3210f8-b1e5-469b-a599-a9dac2603c17 service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:51.707646 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3210f8-b1e5-469b-a599-a9dac2603c17 service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:51.707918 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3210f8-b1e5-469b-a599-a9dac2603c17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:51.718033 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3210f8-b1e5-469b-a599-a9dac2603c17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:51.719449 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3210f8-b1e5-469b-a599-a9dac2603c17 service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:51.741744 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1067/2142] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:47:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:47:52.010021 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b50ef5-56a9-434d-ba68-c148bf16b44e None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:52.011561 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-74b50ef5-56a9-434d-ba68-c148bf16b44e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:52.023948 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b50ef5-56a9-434d-ba68-c148bf16b44e None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:52.030116 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560a4813-ae1c-4fda-b7b3-04fa4de8a130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:52.030761 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560a4813-ae1c-4fda-b7b3-04fa4de8a130 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:52.031042 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560a4813-ae1c-4fda-b7b3-04fa4de8a130 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:52.031311 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560a4813-ae1c-4fda-b7b3-04fa4de8a130 admin admin] PATH_INFO: `/v3/projects/1166971ce3a546e2af32d938e70af473` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:52.032519 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560a4813-ae1c-4fda-b7b3-04fa4de8a130 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1166971ce3a546e2af32d938e70af473)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:52.035322 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560a4813-ae1c-4fda-b7b3-04fa4de8a130 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:52.042840 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:47:52.097627 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-560a4813-ae1c-4fda-b7b3-04fa4de8a130 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1166971ce3a546e2af32d938e70af473', 'request_id': 'req-560a4813-ae1c-4fda-b7b3-04fa4de8a130'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:52.114196 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-560a4813-ae1c-4fda-b7b3-04fa4de8a130 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1166971ce3a546e2af32d938e70af473', 'request_id': 'req-560a4813-ae1c-4fda-b7b3-04fa4de8a130'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:52.114196 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-560a4813-ae1c-4fda-b7b3-04fa4de8a130 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1166971ce3a546e2af32d938e70af473', 'request_id': 'req-560a4813-ae1c-4fda-b7b3-04fa4de8a130'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:52.126524 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-560a4813-ae1c-4fda-b7b3-04fa4de8a130 admin admin] The token cache is being invalidate because project 1166971ce3a546e2af32d938e70af473 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86933) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 13:47:52.126899 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-560a4813-ae1c-4fda-b7b3-04fa4de8a130 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:52.127636 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-560a4813-ae1c-4fda-b7b3-04fa4de8a130 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:47:52.129467 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1076/2143] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 13:47:52 2022] DELETE /identity/v3/projects/1166971ce3a546e2af32d938e70af473 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:47:52.134145 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6478fc3-4476-4fe2-bbc8-e21cfb25284e None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:52.134710 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6478fc3-4476-4fe2-bbc8-e21cfb25284e None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:52.134996 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6478fc3-4476-4fe2-bbc8-e21cfb25284e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:52.159759 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-b6478fc3-4476-4fe2-bbc8-e21cfb25284e None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:47:52.161212 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-b6478fc3-4476-4fe2-bbc8-e21cfb25284e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:52.204406 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:47:52.234262 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1068/2144] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:47:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:47:52.242713 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf65167-37e5-44cb-940b-210a99be06a3 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:52.243067 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf65167-37e5-44cb-940b-210a99be06a3 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:52.243411 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf65167-37e5-44cb-940b-210a99be06a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:52.266454 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-3bf65167-37e5-44cb-940b-210a99be06a3 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:47:52.267770 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-3bf65167-37e5-44cb-940b-210a99be06a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:52.276089 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1077/2145] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:47:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:47:52.281387 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6478fc3-4476-4fe2-bbc8-e21cfb25284e None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:52.292029 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6478fc3-4476-4fe2-bbc8-e21cfb25284e None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:52.297829 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceb2df7-26f7-40fa-b192-1555a7eef0ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:52.298609 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceb2df7-26f7-40fa-b192-1555a7eef0ed admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:52.298895 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceb2df7-26f7-40fa-b192-1555a7eef0ed admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:52.299198 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceb2df7-26f7-40fa-b192-1555a7eef0ed admin admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:52.299925 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dceb2df7-26f7-40fa-b192-1555a7eef0ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:47:52.300263 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dceb2df7-26f7-40fa-b192-1555a7eef0ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:52.301976 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dceb2df7-26f7-40fa-b192-1555a7eef0ed admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:52.315696 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1069/2146] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Jul 26 13:47:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:47:52.319872 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf65167-37e5-44cb-940b-210a99be06a3 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:52.331061 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf65167-37e5-44cb-940b-210a99be06a3 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:52.335871 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20fd2f1-3051-4557-91ba-dd680d9057e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:52.336570 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20fd2f1-3051-4557-91ba-dd680d9057e1 admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:52.336852 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20fd2f1-3051-4557-91ba-dd680d9057e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:52.337127 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20fd2f1-3051-4557-91ba-dd680d9057e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:52.337837 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20fd2f1-3051-4557-91ba-dd680d9057e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:52.339783 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20fd2f1-3051-4557-91ba-dd680d9057e1 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:52.367877 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1078/2147] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 13:47:52 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:47:52.372136 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceb2df7-26f7-40fa-b192-1555a7eef0ed admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:52.383342 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceb2df7-26f7-40fa-b192-1555a7eef0ed admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:52.443651 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16df715-cd52-4b1b-84ff-c09e974a7c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:52.444261 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16df715-cd52-4b1b-84ff-c09e974a7c79 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:52.444537 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16df715-cd52-4b1b-84ff-c09e974a7c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:52.444805 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16df715-cd52-4b1b-84ff-c09e974a7c79 admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:52.445495 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16df715-cd52-4b1b-84ff-c09e974a7c79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:52.447391 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16df715-cd52-4b1b-84ff-c09e974a7c79 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:52.469882 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1070/2148] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 13:47:52 2022] POST /identity/v3/users => generated 289 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 13:47:52.474381 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20fd2f1-3051-4557-91ba-dd680d9057e1 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:52.485434 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20fd2f1-3051-4557-91ba-dd680d9057e1 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:52.490156 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b5d5a6-cd90-4875-ad65-74d699d99915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:52.490770 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b5d5a6-cd90-4875-ad65-74d699d99915 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:52.491056 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b5d5a6-cd90-4875-ad65-74d699d99915 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:52.491335 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b5d5a6-cd90-4875-ad65-74d699d99915 admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:52.492035 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b5d5a6-cd90-4875-ad65-74d699d99915 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:52.493680 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b5d5a6-cd90-4875-ad65-74d699d99915 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:52.504253 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1079/2149] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:47:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:47:52.508171 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16df715-cd52-4b1b-84ff-c09e974a7c79 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:52.518079 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16df715-cd52-4b1b-84ff-c09e974a7c79 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:52.522731 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24b7d78-c7cc-4f32-b44b-bb9439b1e8e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:52.523335 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24b7d78-c7cc-4f32-b44b-bb9439b1e8e7 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:52.523622 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24b7d78-c7cc-4f32-b44b-bb9439b1e8e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:52.523901 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24b7d78-c7cc-4f32-b44b-bb9439b1e8e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:52.524598 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24b7d78-c7cc-4f32-b44b-bb9439b1e8e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:52.526217 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24b7d78-c7cc-4f32-b44b-bb9439b1e8e7 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:52.536605 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1071/2150] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:47:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:47:52.541465 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b5d5a6-cd90-4875-ad65-74d699d99915 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:52.552729 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b5d5a6-cd90-4875-ad65-74d699d99915 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:52.557362 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a43b12-d7b9-406b-90ae-3aa55b736408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:52.558066 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a43b12-d7b9-406b-90ae-3aa55b736408 admin admin] REQUEST_METHOD: `PUT` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:52.558349 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a43b12-d7b9-406b-90ae-3aa55b736408 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:52.558628 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a43b12-d7b9-406b-90ae-3aa55b736408 admin admin] PATH_INFO: `/v3/projects/87ef798b967d4b07b1e100d13edd0fd3/users/816fb8efa96744efbf7648da1f2f4261/roles/759f389b221049c4b01faf3b984cd8d7` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:52.573423 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a43b12-d7b9-406b-90ae-3aa55b736408 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87ef798b967d4b07b1e100d13edd0fd3, user_id=816fb8efa96744efbf7648da1f2f4261, role_id=759f389b221049c4b01faf3b984cd8d7)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:52.576095 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a43b12-d7b9-406b-90ae-3aa55b736408 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:52.590641 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1080/2151] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 13:47:52 2022] PUT /identity/v3/projects/87ef798b967d4b07b1e100d13edd0fd3/users/816fb8efa96744efbf7648da1f2f4261/roles/759f389b221049c4b01faf3b984cd8d7 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:47:52.601044 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63e4a80-1995-42eb-8c72-b4b0459a520c None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:52.601338 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63e4a80-1995-42eb-8c72-b4b0459a520c None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:52.601651 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63e4a80-1995-42eb-8c72-b4b0459a520c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:52.630280 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-b63e4a80-1995-42eb-8c72-b4b0459a520c None None] MFA Rules not processed for user `816fb8efa96744efbf7648da1f2f4261`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:47:52.631507 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-b63e4a80-1995-42eb-8c72-b4b0459a520c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:52.701553 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1072/2152] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:47:52 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:47:52.712930 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a43b12-d7b9-406b-90ae-3aa55b736408 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:47:52.714652 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-66a43b12-d7b9-406b-90ae-3aa55b736408 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:47:52.771549 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a43b12-d7b9-406b-90ae-3aa55b736408 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:47:52.775164 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53f5c3d-d62c-418d-a8ab-d3ffc0304804 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:47:52.775775 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53f5c3d-d62c-418d-a8ab-d3ffc0304804 service cinder] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:47:52.776054 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53f5c3d-d62c-418d-a8ab-d3ffc0304804 service cinder] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:47:52.776325 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53f5c3d-d62c-418d-a8ab-d3ffc0304804 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:47:52.784977 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53f5c3d-d62c-418d-a8ab-d3ffc0304804 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:47:52.786455 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53f5c3d-d62c-418d-a8ab-d3ffc0304804 service cinder] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:47:52.797358 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1081/2153] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:47:52 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:48:02.698517 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=81,keystone:INSERT=3 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:48:02.791428 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=75,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:48:10.882034 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63e4a80-1995-42eb-8c72-b4b0459a520c None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:48:10.883692 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-b63e4a80-1995-42eb-8c72-b4b0459a520c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:48:11.006347 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63e4a80-1995-42eb-8c72-b4b0459a520c None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:48:11.011116 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fd7070-8c3a-434b-b29a-257fdfc5b778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:48:11.011780 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fd7070-8c3a-434b-b29a-257fdfc5b778 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:48:11.012066 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fd7070-8c3a-434b-b29a-257fdfc5b778 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:48:11.012346 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fd7070-8c3a-434b-b29a-257fdfc5b778 admin admin] PATH_INFO: `/v3/users/1f5b2d80af624d45977460439d4eec2b` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:48:11.018221 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fd7070-8c3a-434b-b29a-257fdfc5b778 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f5b2d80af624d45977460439d4eec2b)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:48:11.020273 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fd7070-8c3a-434b-b29a-257fdfc5b778 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:48:11.068503 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-b1fd7070-8c3a-434b-b29a-257fdfc5b778 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f5b2d80af624d45977460439d4eec2b'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:48:11.069156 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-b1fd7070-8c3a-434b-b29a-257fdfc5b778 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f5b2d80af624d45977460439d4eec2b'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:48:11.076580 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-b1fd7070-8c3a-434b-b29a-257fdfc5b778 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f5b2d80af624d45977460439d4eec2b'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:48:11.077198 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-b1fd7070-8c3a-434b-b29a-257fdfc5b778 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f5b2d80af624d45977460439d4eec2b'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:48:11.088892 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-b1fd7070-8c3a-434b-b29a-257fdfc5b778 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f5b2d80af624d45977460439d4eec2b'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:48:11.102035 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1073/2154] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 13:48:10 2022] DELETE /identity/v3/users/1f5b2d80af624d45977460439d4eec2b => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:48:11.275083 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53f5c3d-d62c-418d-a8ab-d3ffc0304804 service cinder] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:48:11.286422 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53f5c3d-d62c-418d-a8ab-d3ffc0304804 service cinder] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:48:11.342889 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574870eb-5d46-4747-9237-8217a6fda770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:48:11.343458 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574870eb-5d46-4747-9237-8217a6fda770 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:48:11.343735 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574870eb-5d46-4747-9237-8217a6fda770 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:48:11.344008 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574870eb-5d46-4747-9237-8217a6fda770 admin admin] PATH_INFO: `/v3/projects/629ae23d3b7b49cba61867a10e372a5c` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:48:11.345163 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574870eb-5d46-4747-9237-8217a6fda770 admin admin] RBAC: Authorizing `identity:delete_project(project_id=629ae23d3b7b49cba61867a10e372a5c)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:48:11.347093 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574870eb-5d46-4747-9237-8217a6fda770 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:48:11.392898 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-574870eb-5d46-4747-9237-8217a6fda770 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '629ae23d3b7b49cba61867a10e372a5c', 'request_id': 'req-574870eb-5d46-4747-9237-8217a6fda770'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:48:11.399691 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-574870eb-5d46-4747-9237-8217a6fda770 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '629ae23d3b7b49cba61867a10e372a5c', 'request_id': 'req-574870eb-5d46-4747-9237-8217a6fda770'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:48:11.405577 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-574870eb-5d46-4747-9237-8217a6fda770 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '629ae23d3b7b49cba61867a10e372a5c', 'request_id': 'req-574870eb-5d46-4747-9237-8217a6fda770'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:48:11.416535 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-574870eb-5d46-4747-9237-8217a6fda770 admin admin] The token cache is being invalidate because project 629ae23d3b7b49cba61867a10e372a5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86933) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 13:48:11.416826 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-574870eb-5d46-4747-9237-8217a6fda770 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:48:11.417673 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-574870eb-5d46-4747-9237-8217a6fda770 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:48:11.419180 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1082/2155] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 13:48:11 2022] DELETE /identity/v3/projects/629ae23d3b7b49cba61867a10e372a5c => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:48:11.424586 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8d2d89-bf1f-4389-bd40-7d1eda38464a None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:48:11.424884 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8d2d89-bf1f-4389-bd40-7d1eda38464a None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:48:11.425167 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8d2d89-bf1f-4389-bd40-7d1eda38464a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:48:11.446794 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-4b8d2d89-bf1f-4389-bd40-7d1eda38464a None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:48:11.448158 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-4b8d2d89-bf1f-4389-bd40-7d1eda38464a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:48:11.518017 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1074/2156] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:48:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:48:11.525957 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599ec294-8412-4e85-85d9-e151249099f9 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:48:11.526251 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599ec294-8412-4e85-85d9-e151249099f9 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:48:11.526530 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599ec294-8412-4e85-85d9-e151249099f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:48:11.547541 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-599ec294-8412-4e85-85d9-e151249099f9 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:48:11.548927 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-599ec294-8412-4e85-85d9-e151249099f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:48:11.556837 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1083/2157] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:48:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:48:11.561435 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8d2d89-bf1f-4389-bd40-7d1eda38464a None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:48:11.571503 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8d2d89-bf1f-4389-bd40-7d1eda38464a None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:48:11.576153 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00133293-89ef-4570-8bb5-ea8f524935c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:48:11.576763 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00133293-89ef-4570-8bb5-ea8f524935c2 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:48:11.577043 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00133293-89ef-4570-8bb5-ea8f524935c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:48:11.577312 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00133293-89ef-4570-8bb5-ea8f524935c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:48:11.578041 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00133293-89ef-4570-8bb5-ea8f524935c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:48:11.578344 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00133293-89ef-4570-8bb5-ea8f524935c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:48:11.580262 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00133293-89ef-4570-8bb5-ea8f524935c2 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:48:11.592447 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1075/2158] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Jul 26 13:48:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:48:11.596772 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599ec294-8412-4e85-85d9-e151249099f9 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:48:11.606985 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599ec294-8412-4e85-85d9-e151249099f9 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:48:11.612138 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5311ac1-e0e3-4812-8a6c-65c5b6d87687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:48:11.612749 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5311ac1-e0e3-4812-8a6c-65c5b6d87687 admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:48:11.613030 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5311ac1-e0e3-4812-8a6c-65c5b6d87687 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:48:11.613302 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5311ac1-e0e3-4812-8a6c-65c5b6d87687 admin admin] PATH_INFO: `/v3/projects` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:48:11.614009 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5311ac1-e0e3-4812-8a6c-65c5b6d87687 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:48:11.615888 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5311ac1-e0e3-4812-8a6c-65c5b6d87687 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:48:11.643910 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1084/2159] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 13:48:11 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:48:11.647894 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00133293-89ef-4570-8bb5-ea8f524935c2 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:48:11.658914 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00133293-89ef-4570-8bb5-ea8f524935c2 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:48:11.662735 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:48:11.727497 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c949c72-cd38-4bf7-9771-22c425d11bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:48:11.728161 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c949c72-cd38-4bf7-9771-22c425d11bad admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:48:11.728446 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c949c72-cd38-4bf7-9771-22c425d11bad admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:48:11.728719 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c949c72-cd38-4bf7-9771-22c425d11bad admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:48:11.729407 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c949c72-cd38-4bf7-9771-22c425d11bad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:48:11.731408 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c949c72-cd38-4bf7-9771-22c425d11bad admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:48:11.753772 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1076/2160] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 13:48:11 2022] POST /identity/v3/users => generated 289 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:48:11.757641 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5311ac1-e0e3-4812-8a6c-65c5b6d87687 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:48:11.768775 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5311ac1-e0e3-4812-8a6c-65c5b6d87687 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:48:11.776050 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7113a308-e2ab-41c1-9bbc-69aedcbea8c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:48:11.776715 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7113a308-e2ab-41c1-9bbc-69aedcbea8c9 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:48:11.777026 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7113a308-e2ab-41c1-9bbc-69aedcbea8c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:48:11.777309 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7113a308-e2ab-41c1-9bbc-69aedcbea8c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:48:11.778095 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7113a308-e2ab-41c1-9bbc-69aedcbea8c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:48:11.779979 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7113a308-e2ab-41c1-9bbc-69aedcbea8c9 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:48:11.790605 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1085/2161] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:48:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:48:11.794725 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c949c72-cd38-4bf7-9771-22c425d11bad admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:48:11.805300 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c949c72-cd38-4bf7-9771-22c425d11bad admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:48:11.810975 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36002c83-51bd-47e1-b266-8c3f0b2697be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:48:11.811602 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36002c83-51bd-47e1-b266-8c3f0b2697be admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:48:11.812010 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36002c83-51bd-47e1-b266-8c3f0b2697be admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:48:11.812292 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36002c83-51bd-47e1-b266-8c3f0b2697be admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:48:11.812997 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36002c83-51bd-47e1-b266-8c3f0b2697be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:48:11.814640 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36002c83-51bd-47e1-b266-8c3f0b2697be admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:48:11.827022 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1077/2162] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:48:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:48:11.831287 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7113a308-e2ab-41c1-9bbc-69aedcbea8c9 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:48:11.841324 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7113a308-e2ab-41c1-9bbc-69aedcbea8c9 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:48:11.845950 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3717fc-e512-4444-b39f-74a7b4d161ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:48:11.846612 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3717fc-e512-4444-b39f-74a7b4d161ea admin admin] REQUEST_METHOD: `PUT` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:48:11.846943 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3717fc-e512-4444-b39f-74a7b4d161ea admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:48:11.847174 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3717fc-e512-4444-b39f-74a7b4d161ea admin admin] PATH_INFO: `/v3/projects/231dc10dd04c4bd39a971979ef1cb7a4/users/bd9ee1f0f2354f048dd6e809121fa0aa/roles/759f389b221049c4b01faf3b984cd8d7` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:48:11.865362 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3717fc-e512-4444-b39f-74a7b4d161ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=231dc10dd04c4bd39a971979ef1cb7a4, user_id=bd9ee1f0f2354f048dd6e809121fa0aa, role_id=759f389b221049c4b01faf3b984cd8d7)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:48:11.868246 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3717fc-e512-4444-b39f-74a7b4d161ea admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:48:11.882577 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1086/2163] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 13:48:11 2022] PUT /identity/v3/projects/231dc10dd04c4bd39a971979ef1cb7a4/users/bd9ee1f0f2354f048dd6e809121fa0aa/roles/759f389b221049c4b01faf3b984cd8d7 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:48:11.887740 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a95b3c-dafa-48f3-a038-59765df80c35 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:48:11.888319 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a95b3c-dafa-48f3-a038-59765df80c35 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:48:11.888416 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a95b3c-dafa-48f3-a038-59765df80c35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:48:11.907743 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-c2a95b3c-dafa-48f3-a038-59765df80c35 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:48:11.909004 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-c2a95b3c-dafa-48f3-a038-59765df80c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:48:11.969391 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1078/2164] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:48:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:48:11.979045 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3717fc-e512-4444-b39f-74a7b4d161ea admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:48:11.980763 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-cf3717fc-e512-4444-b39f-74a7b4d161ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:48:12.012256 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:48:12.047818 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3717fc-e512-4444-b39f-74a7b4d161ea admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:48:12.052194 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b77c7a-3a55-4ef3-b490-ad38bb8bceba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:48:12.052824 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b77c7a-3a55-4ef3-b490-ad38bb8bceba service neutron] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:48:12.053106 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b77c7a-3a55-4ef3-b490-ad38bb8bceba service neutron] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:48:12.060163 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b77c7a-3a55-4ef3-b490-ad38bb8bceba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:48:12.064057 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b77c7a-3a55-4ef3-b490-ad38bb8bceba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:48:12.065595 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b77c7a-3a55-4ef3-b490-ad38bb8bceba service neutron] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:48:12.075170 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1087/2165] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:48:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:48:14.663481 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a95b3c-dafa-48f3-a038-59765df80c35 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:48:14.665093 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-c2a95b3c-dafa-48f3-a038-59765df80c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:48:14.681468 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a95b3c-dafa-48f3-a038-59765df80c35 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:48:14.687533 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84a3ae8-76ff-4141-bd19-ac039099dae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:48:14.688441 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84a3ae8-76ff-4141-bd19-ac039099dae9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:48:14.688916 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84a3ae8-76ff-4141-bd19-ac039099dae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:48:14.689199 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84a3ae8-76ff-4141-bd19-ac039099dae9 admin admin] PATH_INFO: `/v3/users/816fb8efa96744efbf7648da1f2f4261` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:48:14.690442 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84a3ae8-76ff-4141-bd19-ac039099dae9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=816fb8efa96744efbf7648da1f2f4261)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:48:14.692366 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84a3ae8-76ff-4141-bd19-ac039099dae9 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:48:14.754571 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-d84a3ae8-76ff-4141-bd19-ac039099dae9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '816fb8efa96744efbf7648da1f2f4261'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:48:14.755541 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-d84a3ae8-76ff-4141-bd19-ac039099dae9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '816fb8efa96744efbf7648da1f2f4261'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:48:14.766432 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-d84a3ae8-76ff-4141-bd19-ac039099dae9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '816fb8efa96744efbf7648da1f2f4261'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:48:14.767401 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-d84a3ae8-76ff-4141-bd19-ac039099dae9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '816fb8efa96744efbf7648da1f2f4261'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:48:14.787286 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-d84a3ae8-76ff-4141-bd19-ac039099dae9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '816fb8efa96744efbf7648da1f2f4261'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:48:14.798089 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:48:14.809905 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1079/2166] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 13:48:14 2022] DELETE /identity/v3/users/816fb8efa96744efbf7648da1f2f4261 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 13:48:14.811411 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e3a14a-0875-47ce-b69f-08cc929625ef None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:48:14.811981 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e3a14a-0875-47ce-b69f-08cc929625ef None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:48:14.811981 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e3a14a-0875-47ce-b69f-08cc929625ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:48:14.834943 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-31e3a14a-0875-47ce-b69f-08cc929625ef None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:48:14.836706 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-31e3a14a-0875-47ce-b69f-08cc929625ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:48:14.946327 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1088/2167] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:48:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:48:14.971464 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84a3ae8-76ff-4141-bd19-ac039099dae9 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:48:14.971464 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-d84a3ae8-76ff-4141-bd19-ac039099dae9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:48:15.074716 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84a3ae8-76ff-4141-bd19-ac039099dae9 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:48:15.090504 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb510a45-0339-4853-a13e-1691eec14b39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:48:15.092072 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb510a45-0339-4853-a13e-1691eec14b39 service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:48:15.092651 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb510a45-0339-4853-a13e-1691eec14b39 service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:48:15.092780 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb510a45-0339-4853-a13e-1691eec14b39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:48:15.110003 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb510a45-0339-4853-a13e-1691eec14b39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:48:15.112742 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb510a45-0339-4853-a13e-1691eec14b39 service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:48:15.135975 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1080/2168] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:48:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:48:15.632691 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e3a14a-0875-47ce-b69f-08cc929625ef None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:48:15.634665 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-31e3a14a-0875-47ce-b69f-08cc929625ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:48:15.661290 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e3a14a-0875-47ce-b69f-08cc929625ef None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:48:15.676903 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea6fbf2-dc93-433d-bfa6-5869261c5aa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:48:15.678034 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea6fbf2-dc93-433d-bfa6-5869261c5aa6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:48:15.678345 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea6fbf2-dc93-433d-bfa6-5869261c5aa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:48:15.680419 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea6fbf2-dc93-433d-bfa6-5869261c5aa6 admin admin] PATH_INFO: `/v3/projects/87ef798b967d4b07b1e100d13edd0fd3` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:48:15.680419 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea6fbf2-dc93-433d-bfa6-5869261c5aa6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87ef798b967d4b07b1e100d13edd0fd3)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:48:15.681919 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea6fbf2-dc93-433d-bfa6-5869261c5aa6 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:48:15.784865 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-3ea6fbf2-dc93-433d-bfa6-5869261c5aa6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87ef798b967d4b07b1e100d13edd0fd3', 'request_id': 'req-3ea6fbf2-dc93-433d-bfa6-5869261c5aa6'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:48:15.797371 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-3ea6fbf2-dc93-433d-bfa6-5869261c5aa6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87ef798b967d4b07b1e100d13edd0fd3', 'request_id': 'req-3ea6fbf2-dc93-433d-bfa6-5869261c5aa6'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:48:15.811053 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-3ea6fbf2-dc93-433d-bfa6-5869261c5aa6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87ef798b967d4b07b1e100d13edd0fd3', 'request_id': 'req-3ea6fbf2-dc93-433d-bfa6-5869261c5aa6'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:48:15.842374 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-3ea6fbf2-dc93-433d-bfa6-5869261c5aa6 admin admin] The token cache is being invalidate because project 87ef798b967d4b07b1e100d13edd0fd3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86933) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 13:48:15.843314 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-3ea6fbf2-dc93-433d-bfa6-5869261c5aa6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:48:15.844728 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-3ea6fbf2-dc93-433d-bfa6-5869261c5aa6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:48:15.847408 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1089/2169] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 13:48:15 2022] DELETE /identity/v3/projects/87ef798b967d4b07b1e100d13edd0fd3 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:48:15.852002 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a559d2a-5c77-4000-a8f2-1ad2afdd9de2 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:48:15.852940 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a559d2a-5c77-4000-a8f2-1ad2afdd9de2 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:48:15.853381 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a559d2a-5c77-4000-a8f2-1ad2afdd9de2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:48:15.896741 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-7a559d2a-5c77-4000-a8f2-1ad2afdd9de2 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:48:15.898788 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-7a559d2a-5c77-4000-a8f2-1ad2afdd9de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:48:15.999429 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1081/2170] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:48:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:48:16.000645 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63df3782-bb01-4826-bef0-4daf98106f4a None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:48:16.001394 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63df3782-bb01-4826-bef0-4daf98106f4a None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:48:16.001781 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63df3782-bb01-4826-bef0-4daf98106f4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:48:16.026939 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-63df3782-bb01-4826-bef0-4daf98106f4a None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:48:16.043124 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-63df3782-bb01-4826-bef0-4daf98106f4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:48:16.059735 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1090/2171] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:48:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 13:48:16.061021 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a559d2a-5c77-4000-a8f2-1ad2afdd9de2 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:48:16.076289 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a559d2a-5c77-4000-a8f2-1ad2afdd9de2 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:48:16.082031 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa875710-20c3-4639-9545-982bb281c2a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:48:16.082909 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa875710-20c3-4639-9545-982bb281c2a6 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:48:16.083323 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa875710-20c3-4639-9545-982bb281c2a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:48:16.083704 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa875710-20c3-4639-9545-982bb281c2a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:48:16.084776 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa875710-20c3-4639-9545-982bb281c2a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:48:16.085190 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa875710-20c3-4639-9545-982bb281c2a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:48:16.086990 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa875710-20c3-4639-9545-982bb281c2a6 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:48:16.109225 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1082/2172] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Jul 26 13:48:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:48:16.113616 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63df3782-bb01-4826-bef0-4daf98106f4a None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:48:16.131403 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63df3782-bb01-4826-bef0-4daf98106f4a None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:48:16.137019 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e367d865-a3b4-4a8e-85f3-db882dbf4118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:48:16.138302 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e367d865-a3b4-4a8e-85f3-db882dbf4118 admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:48:16.138749 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e367d865-a3b4-4a8e-85f3-db882dbf4118 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:48:16.139360 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e367d865-a3b4-4a8e-85f3-db882dbf4118 admin admin] PATH_INFO: `/v3/projects` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:48:16.140373 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e367d865-a3b4-4a8e-85f3-db882dbf4118 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:48:16.142994 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e367d865-a3b4-4a8e-85f3-db882dbf4118 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:48:16.212324 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1091/2173] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 13:48:16 2022] POST /identity/v3/projects => generated 375 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:48:16.215739 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa875710-20c3-4639-9545-982bb281c2a6 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:48:16.230501 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa875710-20c3-4639-9545-982bb281c2a6 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:48:16.332222 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c910a9b-45ac-4531-92b9-d1bd0efff1c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:48:16.332875 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c910a9b-45ac-4531-92b9-d1bd0efff1c3 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:48:16.333157 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c910a9b-45ac-4531-92b9-d1bd0efff1c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:48:16.333430 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c910a9b-45ac-4531-92b9-d1bd0efff1c3 admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:48:16.334133 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c910a9b-45ac-4531-92b9-d1bd0efff1c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:48:16.336076 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c910a9b-45ac-4531-92b9-d1bd0efff1c3 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:48:16.370326 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1083/2174] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 13:48:16 2022] POST /identity/v3/users => generated 288 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:48:16.377461 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e367d865-a3b4-4a8e-85f3-db882dbf4118 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:48:16.396317 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e367d865-a3b4-4a8e-85f3-db882dbf4118 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:48:16.416469 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb62be21-ee78-4500-aa79-2036c50bd393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:48:16.417800 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb62be21-ee78-4500-aa79-2036c50bd393 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:48:16.417800 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb62be21-ee78-4500-aa79-2036c50bd393 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:48:16.418055 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb62be21-ee78-4500-aa79-2036c50bd393 admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:48:16.419019 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb62be21-ee78-4500-aa79-2036c50bd393 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:48:16.421592 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb62be21-ee78-4500-aa79-2036c50bd393 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:48:16.454059 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1092/2175] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:48:16 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:48:16.459023 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c910a9b-45ac-4531-92b9-d1bd0efff1c3 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:48:16.472651 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c910a9b-45ac-4531-92b9-d1bd0efff1c3 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:48:16.494006 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e047340-f168-437b-bd6b-e0b923d48008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:48:16.494006 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e047340-f168-437b-bd6b-e0b923d48008 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:48:16.494006 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e047340-f168-437b-bd6b-e0b923d48008 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:48:16.494006 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e047340-f168-437b-bd6b-e0b923d48008 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:48:16.494357 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e047340-f168-437b-bd6b-e0b923d48008 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:48:16.494357 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e047340-f168-437b-bd6b-e0b923d48008 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:48:16.524670 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1084/2176] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:48:16 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:48:16.526632 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb62be21-ee78-4500-aa79-2036c50bd393 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:48:16.550626 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb62be21-ee78-4500-aa79-2036c50bd393 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:48:16.564636 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cff9ad-6935-444a-ae31-7ff5f155f495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:48:16.565346 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cff9ad-6935-444a-ae31-7ff5f155f495 admin admin] REQUEST_METHOD: `PUT` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:48:16.565722 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cff9ad-6935-444a-ae31-7ff5f155f495 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:48:16.566012 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cff9ad-6935-444a-ae31-7ff5f155f495 admin admin] PATH_INFO: `/v3/projects/d6649531b6f24f7f94c1a1740941f710/users/ea1cd5eb9b0d42e1a94344dcf3e0f180/roles/759f389b221049c4b01faf3b984cd8d7` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:48:16.588023 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:48:16.602753 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cff9ad-6935-444a-ae31-7ff5f155f495 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6649531b6f24f7f94c1a1740941f710, user_id=ea1cd5eb9b0d42e1a94344dcf3e0f180, role_id=759f389b221049c4b01faf3b984cd8d7)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:48:16.608383 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cff9ad-6935-444a-ae31-7ff5f155f495 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:48:16.647436 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1093/2177] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 13:48:16 2022] PUT /identity/v3/projects/d6649531b6f24f7f94c1a1740941f710/users/ea1cd5eb9b0d42e1a94344dcf3e0f180/roles/759f389b221049c4b01faf3b984cd8d7 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:48:16.649229 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adfc09f-ced9-4611-8a3e-9ce408703b12 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:48:16.649569 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adfc09f-ced9-4611-8a3e-9ce408703b12 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:48:16.649866 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adfc09f-ced9-4611-8a3e-9ce408703b12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:48:16.665029 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:48:16.698419 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-5adfc09f-ced9-4611-8a3e-9ce408703b12 None None] MFA Rules not processed for user `ea1cd5eb9b0d42e1a94344dcf3e0f180`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:48:16.698419 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-5adfc09f-ced9-4611-8a3e-9ce408703b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:48:16.703469 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a86ba7-ac8a-4326-b542-26ba6726483c None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:48:16.703773 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a86ba7-ac8a-4326-b542-26ba6726483c None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:48:16.704130 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a86ba7-ac8a-4326-b542-26ba6726483c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:48:16.753682 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-b3a86ba7-ac8a-4326-b542-26ba6726483c None None] MFA Rules not processed for user `bd9ee1f0f2354f048dd6e809121fa0aa`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:48:16.755340 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-b3a86ba7-ac8a-4326-b542-26ba6726483c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:48:16.786577 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1085/2178] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:48:16 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:48:16.787491 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adfc09f-ced9-4611-8a3e-9ce408703b12 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:48:16.867952 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1094/2179] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:48:16 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:48:16.868723 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a86ba7-ac8a-4326-b542-26ba6726483c None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:48:16.877129 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-b3a86ba7-ac8a-4326-b542-26ba6726483c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:48:16.934726 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adfc09f-ced9-4611-8a3e-9ce408703b12 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:48:16.951945 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1758ed-6cca-4d8d-919e-51fde14a2f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:48:16.952912 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1758ed-6cca-4d8d-919e-51fde14a2f71 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:48:16.953417 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1758ed-6cca-4d8d-919e-51fde14a2f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:48:16.954159 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1758ed-6cca-4d8d-919e-51fde14a2f71 admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:48:16.955228 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1758ed-6cca-4d8d-919e-51fde14a2f71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:48:16.957502 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1758ed-6cca-4d8d-919e-51fde14a2f71 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:48:16.968333 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a86ba7-ac8a-4326-b542-26ba6726483c None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:48:16.975629 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e4e3d0-a953-4ba4-8c68-2f06dac7c5a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c69161cd520b4b29bf50f3bf2f1c033b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:48:16.976280 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e4e3d0-a953-4ba4-8c68-2f06dac7c5a4 service nova] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:48:16.976561 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e4e3d0-a953-4ba4-8c68-2f06dac7c5a4 service nova] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:48:16.976837 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e4e3d0-a953-4ba4-8c68-2f06dac7c5a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:48:16.987769 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e4e3d0-a953-4ba4-8c68-2f06dac7c5a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:48:16.989554 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e4e3d0-a953-4ba4-8c68-2f06dac7c5a4 service nova] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:48:16.996955 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1086/2180] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 13:48:16 2022] POST /identity/v3/projects => generated 377 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:48:16.999534 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1758ed-6cca-4d8d-919e-51fde14a2f71 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:48:17.005672 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1095/2181] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:48:16 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:48:17.121165 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1758ed-6cca-4d8d-919e-51fde14a2f71 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:48:17.129185 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a543be14-baea-4ad8-9438-528944758be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:48:17.130350 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a543be14-baea-4ad8-9438-528944758be0 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:48:17.130868 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a543be14-baea-4ad8-9438-528944758be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:48:17.131193 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a543be14-baea-4ad8-9438-528944758be0 admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:48:17.131975 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a543be14-baea-4ad8-9438-528944758be0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:48:17.134672 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a543be14-baea-4ad8-9438-528944758be0 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:48:17.166269 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1087/2182] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 13:48:16 2022] POST /identity/v3/users => generated 289 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:48:17.167527 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e4e3d0-a953-4ba4-8c68-2f06dac7c5a4 service nova] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:48:17.202080 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e4e3d0-a953-4ba4-8c68-2f06dac7c5a4 service nova] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:48:17.208740 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4362f276-e822-47c8-9fe9-40c694d7bf94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:48:17.210133 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4362f276-e822-47c8-9fe9-40c694d7bf94 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:48:17.210681 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4362f276-e822-47c8-9fe9-40c694d7bf94 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:48:17.211067 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4362f276-e822-47c8-9fe9-40c694d7bf94 admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:48:17.212027 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4362f276-e822-47c8-9fe9-40c694d7bf94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:48:17.213863 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4362f276-e822-47c8-9fe9-40c694d7bf94 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:48:17.235522 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1096/2183] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:48:17 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 13:48:17.236791 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a543be14-baea-4ad8-9438-528944758be0 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:48:17.265967 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a543be14-baea-4ad8-9438-528944758be0 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:48:17.272651 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61228a3-388a-4793-90e3-401993d7b01a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:48:17.273452 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61228a3-388a-4793-90e3-401993d7b01a admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:48:17.273775 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61228a3-388a-4793-90e3-401993d7b01a admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:48:17.274065 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61228a3-388a-4793-90e3-401993d7b01a admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:48:17.274918 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61228a3-388a-4793-90e3-401993d7b01a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:48:17.276719 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61228a3-388a-4793-90e3-401993d7b01a admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:48:17.283787 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:48:17.293452 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4362f276-e822-47c8-9fe9-40c694d7bf94 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:48:17.297852 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1088/2184] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:48:17 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:48:17.305617 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4362f276-e822-47c8-9fe9-40c694d7bf94 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:48:17.312064 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95b1187-0d56-4d09-bfa4-fe1de96a29fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:48:17.313078 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95b1187-0d56-4d09-bfa4-fe1de96a29fd admin admin] REQUEST_METHOD: `PUT` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:48:17.313428 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95b1187-0d56-4d09-bfa4-fe1de96a29fd admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:48:17.313778 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95b1187-0d56-4d09-bfa4-fe1de96a29fd admin admin] PATH_INFO: `/v3/projects/fd90c908924b4490a1bc3273f5832cbc/users/c8f96cdc262c4638a66604b88cbb77b3/roles/759f389b221049c4b01faf3b984cd8d7` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:48:17.339193 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95b1187-0d56-4d09-bfa4-fe1de96a29fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd90c908924b4490a1bc3273f5832cbc, user_id=c8f96cdc262c4638a66604b88cbb77b3, role_id=759f389b221049c4b01faf3b984cd8d7)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:48:17.342926 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95b1187-0d56-4d09-bfa4-fe1de96a29fd admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:48:17.360553 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1097/2185] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 13:48:17 2022] PUT /identity/v3/projects/fd90c908924b4490a1bc3273f5832cbc/users/c8f96cdc262c4638a66604b88cbb77b3/roles/759f389b221049c4b01faf3b984cd8d7 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 13:48:17.371141 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed28e8e2-d902-44bd-a85e-a4704bd93acd None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:48:17.371493 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed28e8e2-d902-44bd-a85e-a4704bd93acd None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:48:17.371704 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed28e8e2-d902-44bd-a85e-a4704bd93acd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:48:17.408373 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-ed28e8e2-d902-44bd-a85e-a4704bd93acd None None] MFA Rules not processed for user `c8f96cdc262c4638a66604b88cbb77b3`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:48:17.410362 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-ed28e8e2-d902-44bd-a85e-a4704bd93acd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:48:17.507405 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1089/2186] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:48:17 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:48:17.512345 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95b1187-0d56-4d09-bfa4-fe1de96a29fd admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:48:17.571903 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:48:17.616844 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95b1187-0d56-4d09-bfa4-fe1de96a29fd admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:48:17.630866 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608208b9-3fad-4240-b95d-6787c46a134f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:48:17.631900 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608208b9-3fad-4240-b95d-6787c46a134f admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:48:17.632185 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608208b9-3fad-4240-b95d-6787c46a134f admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:48:17.632456 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608208b9-3fad-4240-b95d-6787c46a134f admin admin] PATH_INFO: `/v3/projects` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:48:17.633860 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608208b9-3fad-4240-b95d-6787c46a134f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:48:17.636840 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608208b9-3fad-4240-b95d-6787c46a134f admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:48:17.689089 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1098/2187] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 13:48:17 2022] POST /identity/v3/projects => generated 377 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:48:17.692874 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed28e8e2-d902-44bd-a85e-a4704bd93acd None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:48:17.705154 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed28e8e2-d902-44bd-a85e-a4704bd93acd None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:48:17.808517 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b75fc5c-7447-4d4e-84c0-107d2b86cee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:48:17.809377 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b75fc5c-7447-4d4e-84c0-107d2b86cee2 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:48:17.809737 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b75fc5c-7447-4d4e-84c0-107d2b86cee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:48:17.810135 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b75fc5c-7447-4d4e-84c0-107d2b86cee2 admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:48:17.810960 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b75fc5c-7447-4d4e-84c0-107d2b86cee2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:48:17.813338 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b75fc5c-7447-4d4e-84c0-107d2b86cee2 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:48:17.839906 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1090/2188] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 13:48:17 2022] POST /identity/v3/users => generated 295 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 13:48:17.844569 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608208b9-3fad-4240-b95d-6787c46a134f admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:48:17.855780 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608208b9-3fad-4240-b95d-6787c46a134f admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:48:17.861174 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce38a3f9-d4da-41ce-bac9-a8a7017df1d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:48:17.861827 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce38a3f9-d4da-41ce-bac9-a8a7017df1d3 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:48:17.862110 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce38a3f9-d4da-41ce-bac9-a8a7017df1d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:48:17.862450 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce38a3f9-d4da-41ce-bac9-a8a7017df1d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:48:17.863369 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce38a3f9-d4da-41ce-bac9-a8a7017df1d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:48:17.864985 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce38a3f9-d4da-41ce-bac9-a8a7017df1d3 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:48:17.877064 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1099/2189] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:48:17 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 13:48:17.882657 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b75fc5c-7447-4d4e-84c0-107d2b86cee2 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:48:17.898883 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b75fc5c-7447-4d4e-84c0-107d2b86cee2 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:48:17.905454 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0f5582-cd81-4154-92bb-59dc9cca29d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:48:17.906396 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0f5582-cd81-4154-92bb-59dc9cca29d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:48:17.906695 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0f5582-cd81-4154-92bb-59dc9cca29d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:48:17.906974 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0f5582-cd81-4154-92bb-59dc9cca29d2 admin admin] PATH_INFO: `/v3/projects/9c7da2007e014012847422658cee1fb8/users/0384fa2d1ffd4468bb74cb3a5d8c8df3/roles/6e0201a9a6484f7d8b112b1eafff70bd` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:48:17.924800 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0f5582-cd81-4154-92bb-59dc9cca29d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c7da2007e014012847422658cee1fb8, user_id=0384fa2d1ffd4468bb74cb3a5d8c8df3, role_id=6e0201a9a6484f7d8b112b1eafff70bd)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:48:17.927765 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0f5582-cd81-4154-92bb-59dc9cca29d2 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:48:17.947154 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1091/2190] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 13:48:17 2022] PUT /identity/v3/projects/9c7da2007e014012847422658cee1fb8/users/0384fa2d1ffd4468bb74cb3a5d8c8df3/roles/6e0201a9a6484f7d8b112b1eafff70bd => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 13:48:17.951341 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce38a3f9-d4da-41ce-bac9-a8a7017df1d3 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:48:17.967939 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce38a3f9-d4da-41ce-bac9-a8a7017df1d3 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:48:18.060478 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875f4c7e-4719-4d91-a0ad-18425f1f2b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:48:18.061221 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875f4c7e-4719-4d91-a0ad-18425f1f2b87 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:48:18.061579 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875f4c7e-4719-4d91-a0ad-18425f1f2b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:48:18.062111 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875f4c7e-4719-4d91-a0ad-18425f1f2b87 admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:48:18.063058 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875f4c7e-4719-4d91-a0ad-18425f1f2b87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:48:18.064661 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875f4c7e-4719-4d91-a0ad-18425f1f2b87 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:48:18.076798 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1100/2191] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:48:17 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:48:18.081182 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0f5582-cd81-4154-92bb-59dc9cca29d2 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:48:18.094834 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0f5582-cd81-4154-92bb-59dc9cca29d2 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:48:18.101569 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbad01a-6cc8-461f-92b1-d5bf33444e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:48:18.102326 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbad01a-6cc8-461f-92b1-d5bf33444e5d admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:48:18.102796 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbad01a-6cc8-461f-92b1-d5bf33444e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:48:18.103082 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbad01a-6cc8-461f-92b1-d5bf33444e5d admin admin] PATH_INFO: `/v3/projects/9c7da2007e014012847422658cee1fb8/users/0384fa2d1ffd4468bb74cb3a5d8c8df3/roles/6e0201a9a6484f7d8b112b1eafff70bd` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:48:18.106523 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbad01a-6cc8-461f-92b1-d5bf33444e5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c7da2007e014012847422658cee1fb8, user_id=0384fa2d1ffd4468bb74cb3a5d8c8df3, role_id=6e0201a9a6484f7d8b112b1eafff70bd)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:48:18.110643 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbad01a-6cc8-461f-92b1-d5bf33444e5d admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:48:18.131199 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1092/2192] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 13:48:18 2022] PUT /identity/v3/projects/9c7da2007e014012847422658cee1fb8/users/0384fa2d1ffd4468bb74cb3a5d8c8df3/roles/6e0201a9a6484f7d8b112b1eafff70bd => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:48:18.145883 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dd0994-bdc1-47af-9d52-b688db434cec None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:48:18.146435 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dd0994-bdc1-47af-9d52-b688db434cec None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:48:18.147011 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dd0994-bdc1-47af-9d52-b688db434cec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:48:18.179849 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-84dd0994-bdc1-47af-9d52-b688db434cec None None] MFA Rules not processed for user `0384fa2d1ffd4468bb74cb3a5d8c8df3`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:48:18.181223 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-84dd0994-bdc1-47af-9d52-b688db434cec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:48:18.292053 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1101/2193] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:48:18 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:48:18.306692 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbad01a-6cc8-461f-92b1-d5bf33444e5d admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:48:18.308676 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-acbad01a-6cc8-461f-92b1-d5bf33444e5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:48:18.375152 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:48:18.387632 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbad01a-6cc8-461f-92b1-d5bf33444e5d admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:48:18.391370 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a126123b-7c6b-4d00-b5ba-f2b685f361ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:48:18.391986 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a126123b-7c6b-4d00-b5ba-f2b685f361ed service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:48:18.392313 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a126123b-7c6b-4d00-b5ba-f2b685f361ed service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:48:18.392593 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a126123b-7c6b-4d00-b5ba-f2b685f361ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:48:18.452342 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a126123b-7c6b-4d00-b5ba-f2b685f361ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:48:18.453864 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a126123b-7c6b-4d00-b5ba-f2b685f361ed service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:48:18.463776 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1093/2194] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:48:18 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:48:18.651748 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc34bb3-c56b-457f-86fa-6c62c0ec99d5 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:48:18.652235 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc34bb3-c56b-457f-86fa-6c62c0ec99d5 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:48:18.652513 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc34bb3-c56b-457f-86fa-6c62c0ec99d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:48:18.674326 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-afc34bb3-c56b-457f-86fa-6c62c0ec99d5 None None] MFA Rules not processed for user `bd9ee1f0f2354f048dd6e809121fa0aa`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:48:18.675751 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-afc34bb3-c56b-457f-86fa-6c62c0ec99d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:48:18.699986 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:48:18.752184 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1102/2195] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:48:18 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:48:20.058418 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a126123b-7c6b-4d00-b5ba-f2b685f361ed service cinder] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:48:20.060295 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-a126123b-7c6b-4d00-b5ba-f2b685f361ed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:48:20.077559 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a126123b-7c6b-4d00-b5ba-f2b685f361ed service cinder] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:48:20.087292 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1d4243-0070-487f-8454-41709bfc31c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:48:20.088759 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1d4243-0070-487f-8454-41709bfc31c6 service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:48:20.089156 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1d4243-0070-487f-8454-41709bfc31c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:48:20.089580 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1d4243-0070-487f-8454-41709bfc31c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:48:20.191608 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1d4243-0070-487f-8454-41709bfc31c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:48:20.193649 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1d4243-0070-487f-8454-41709bfc31c6 service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:48:20.208119 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1094/2196] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:48:20 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:48:21.163510 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc34bb3-c56b-457f-86fa-6c62c0ec99d5 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:48:21.175474 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc34bb3-c56b-457f-86fa-6c62c0ec99d5 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:48:21.180564 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558039a3-a0a0-47aa-aabd-29131b6dfb0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:48:21.181568 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558039a3-a0a0-47aa-aabd-29131b6dfb0c service cinder] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:48:21.182232 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558039a3-a0a0-47aa-aabd-29131b6dfb0c service cinder] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:48:21.183585 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558039a3-a0a0-47aa-aabd-29131b6dfb0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:48:21.199213 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558039a3-a0a0-47aa-aabd-29131b6dfb0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:48:21.201292 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558039a3-a0a0-47aa-aabd-29131b6dfb0c service cinder] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:48:21.213201 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1103/2197] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:48:21 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 13:48:21.238324 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1d4243-0070-487f-8454-41709bfc31c6 service cinder] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:48:21.240068 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-ca1d4243-0070-487f-8454-41709bfc31c6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:48:21.374515 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1d4243-0070-487f-8454-41709bfc31c6 service cinder] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:48:21.392579 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3777dfe5-dc5c-4791-a4c0-e615a0a47e82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:48:21.394088 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3777dfe5-dc5c-4791-a4c0-e615a0a47e82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:48:21.394088 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3777dfe5-dc5c-4791-a4c0-e615a0a47e82 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:48:21.394088 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3777dfe5-dc5c-4791-a4c0-e615a0a47e82 admin admin] PATH_INFO: `/v3/users/26a3ea88c36c42048548a8d46804d3c8` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:48:21.395361 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3777dfe5-dc5c-4791-a4c0-e615a0a47e82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26a3ea88c36c42048548a8d46804d3c8)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:48:21.397820 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3777dfe5-dc5c-4791-a4c0-e615a0a47e82 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:48:21.452588 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:48:21.468536 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-3777dfe5-dc5c-4791-a4c0-e615a0a47e82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26a3ea88c36c42048548a8d46804d3c8'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:48:21.470159 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-3777dfe5-dc5c-4791-a4c0-e615a0a47e82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26a3ea88c36c42048548a8d46804d3c8'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:48:21.480392 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-3777dfe5-dc5c-4791-a4c0-e615a0a47e82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26a3ea88c36c42048548a8d46804d3c8'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:48:21.481725 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-3777dfe5-dc5c-4791-a4c0-e615a0a47e82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26a3ea88c36c42048548a8d46804d3c8'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:48:21.498342 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-3777dfe5-dc5c-4791-a4c0-e615a0a47e82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26a3ea88c36c42048548a8d46804d3c8'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:48:21.520441 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1095/2198] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 13:48:21 2022] DELETE /identity/v3/users/26a3ea88c36c42048548a8d46804d3c8 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:48:21.524921 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2a3997-f34f-4a97-b9cd-f59acc7e41c6 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:48:21.525212 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2a3997-f34f-4a97-b9cd-f59acc7e41c6 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:48:21.525537 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2a3997-f34f-4a97-b9cd-f59acc7e41c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:48:21.546454 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-7b2a3997-f34f-4a97-b9cd-f59acc7e41c6 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:48:21.547697 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-7b2a3997-f34f-4a97-b9cd-f59acc7e41c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:48:21.642590 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1104/2199] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:48:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:48:21.652714 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3777dfe5-dc5c-4791-a4c0-e615a0a47e82 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:48:21.654647 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-3777dfe5-dc5c-4791-a4c0-e615a0a47e82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:48:21.730205 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3777dfe5-dc5c-4791-a4c0-e615a0a47e82 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:48:21.735450 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2230be9c-70c7-4cd9-a866-1ed90037739c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:48:21.736381 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2230be9c-70c7-4cd9-a866-1ed90037739c service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:48:21.736760 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2230be9c-70c7-4cd9-a866-1ed90037739c service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:48:21.737110 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2230be9c-70c7-4cd9-a866-1ed90037739c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:48:21.753447 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2230be9c-70c7-4cd9-a866-1ed90037739c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:48:21.755050 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2230be9c-70c7-4cd9-a866-1ed90037739c service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:48:21.769120 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1096/2200] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:48:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:48:22.295611 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2a3997-f34f-4a97-b9cd-f59acc7e41c6 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:48:22.297889 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-7b2a3997-f34f-4a97-b9cd-f59acc7e41c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:48:22.313391 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2a3997-f34f-4a97-b9cd-f59acc7e41c6 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:48:22.320081 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340113eb-2d24-493f-ae88-65ea1c460b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:48:22.321125 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340113eb-2d24-493f-ae88-65ea1c460b15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:48:22.321646 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340113eb-2d24-493f-ae88-65ea1c460b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:48:22.322911 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340113eb-2d24-493f-ae88-65ea1c460b15 admin admin] PATH_INFO: `/v3/projects/b708762c50314fa89fc955a4eff2ab99` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:48:22.324217 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340113eb-2d24-493f-ae88-65ea1c460b15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b708762c50314fa89fc955a4eff2ab99)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:48:22.326671 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340113eb-2d24-493f-ae88-65ea1c460b15 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:48:22.397988 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-340113eb-2d24-493f-ae88-65ea1c460b15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b708762c50314fa89fc955a4eff2ab99', 'request_id': 'req-340113eb-2d24-493f-ae88-65ea1c460b15'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:48:22.411617 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-340113eb-2d24-493f-ae88-65ea1c460b15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b708762c50314fa89fc955a4eff2ab99', 'request_id': 'req-340113eb-2d24-493f-ae88-65ea1c460b15'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:48:22.435686 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-340113eb-2d24-493f-ae88-65ea1c460b15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b708762c50314fa89fc955a4eff2ab99', 'request_id': 'req-340113eb-2d24-493f-ae88-65ea1c460b15'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:48:22.451893 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-340113eb-2d24-493f-ae88-65ea1c460b15 admin admin] The token cache is being invalidate because project b708762c50314fa89fc955a4eff2ab99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86933) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 13:48:22.452198 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-340113eb-2d24-493f-ae88-65ea1c460b15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:48:22.452885 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-340113eb-2d24-493f-ae88-65ea1c460b15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:48:22.454597 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1105/2201] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 13:48:22 2022] DELETE /identity/v3/projects/b708762c50314fa89fc955a4eff2ab99 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:48:22.459861 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ad9d34-1f49-410f-8fc6-17b194bedf8b None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:48:22.460786 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ad9d34-1f49-410f-8fc6-17b194bedf8b None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:48:22.461098 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ad9d34-1f49-410f-8fc6-17b194bedf8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:48:22.487287 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-52ad9d34-1f49-410f-8fc6-17b194bedf8b None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:48:22.488516 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-52ad9d34-1f49-410f-8fc6-17b194bedf8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:48:22.584970 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1097/2202] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:48:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:48:22.598664 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2764ac-9d8e-44e7-b5a1-a8e70ff27949 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:48:22.598996 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2764ac-9d8e-44e7-b5a1-a8e70ff27949 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:48:22.599442 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2764ac-9d8e-44e7-b5a1-a8e70ff27949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:48:22.636784 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-9a2764ac-9d8e-44e7-b5a1-a8e70ff27949 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:48:22.638405 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-9a2764ac-9d8e-44e7-b5a1-a8e70ff27949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:48:22.649557 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1106/2203] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:48:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:48:22.654481 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ad9d34-1f49-410f-8fc6-17b194bedf8b None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:48:22.675022 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ad9d34-1f49-410f-8fc6-17b194bedf8b None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:48:22.682037 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135419e5-01ce-4098-ae00-096490ae61cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:48:22.682846 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135419e5-01ce-4098-ae00-096490ae61cf admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:48:22.683279 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135419e5-01ce-4098-ae00-096490ae61cf admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:48:22.683718 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135419e5-01ce-4098-ae00-096490ae61cf admin admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:48:22.684600 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135419e5-01ce-4098-ae00-096490ae61cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:48:22.685686 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135419e5-01ce-4098-ae00-096490ae61cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:48:22.687317 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135419e5-01ce-4098-ae00-096490ae61cf admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:48:22.701401 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1098/2204] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Jul 26 13:48:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 13:48:22.710706 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2764ac-9d8e-44e7-b5a1-a8e70ff27949 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:48:22.722206 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2764ac-9d8e-44e7-b5a1-a8e70ff27949 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:48:22.738252 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3a17e1-b678-4fd5-b5ba-db16f34d1e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:48:22.739070 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3a17e1-b678-4fd5-b5ba-db16f34d1e54 admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:48:22.740069 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3a17e1-b678-4fd5-b5ba-db16f34d1e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:48:22.740465 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3a17e1-b678-4fd5-b5ba-db16f34d1e54 admin admin] PATH_INFO: `/v3/projects` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:48:22.741278 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3a17e1-b678-4fd5-b5ba-db16f34d1e54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:48:22.744249 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3a17e1-b678-4fd5-b5ba-db16f34d1e54 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:48:22.793100 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1107/2205] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 13:48:22 2022] POST /identity/v3/projects => generated 397 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:48:22.798843 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135419e5-01ce-4098-ae00-096490ae61cf admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:48:22.815032 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135419e5-01ce-4098-ae00-096490ae61cf admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:48:22.902922 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:48:22.928289 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486c6685-b844-445a-bc10-d72132bd1dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:48:22.928952 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486c6685-b844-445a-bc10-d72132bd1dd5 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:48:22.929239 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486c6685-b844-445a-bc10-d72132bd1dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:48:22.929549 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486c6685-b844-445a-bc10-d72132bd1dd5 admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:48:22.930257 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486c6685-b844-445a-bc10-d72132bd1dd5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:48:22.932369 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486c6685-b844-445a-bc10-d72132bd1dd5 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:48:22.960507 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1099/2206] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 13:48:22 2022] POST /identity/v3/users => generated 299 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:48:22.964500 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3a17e1-b678-4fd5-b5ba-db16f34d1e54 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:48:22.976801 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3a17e1-b678-4fd5-b5ba-db16f34d1e54 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:48:22.982705 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d675f4-9038-4595-973b-8abe1f0655c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:48:22.983520 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d675f4-9038-4595-973b-8abe1f0655c4 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:48:22.983820 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d675f4-9038-4595-973b-8abe1f0655c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:48:22.984101 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d675f4-9038-4595-973b-8abe1f0655c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:48:22.984804 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d675f4-9038-4595-973b-8abe1f0655c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:48:22.986440 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d675f4-9038-4595-973b-8abe1f0655c4 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:48:22.992910 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:48:23.001547 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1108/2207] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:48:22 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:48:23.003021 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486c6685-b844-445a-bc10-d72132bd1dd5 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:48:23.013672 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486c6685-b844-445a-bc10-d72132bd1dd5 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:48:23.020322 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6168886e-c6f9-4922-9ad9-7dd0db553ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:48:23.020958 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6168886e-c6f9-4922-9ad9-7dd0db553ed3 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:48:23.021244 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6168886e-c6f9-4922-9ad9-7dd0db553ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:48:23.021712 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6168886e-c6f9-4922-9ad9-7dd0db553ed3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:48:23.022429 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6168886e-c6f9-4922-9ad9-7dd0db553ed3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:48:23.024031 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6168886e-c6f9-4922-9ad9-7dd0db553ed3 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:48:23.045673 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1100/2208] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:48:23 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:48:23.047090 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d675f4-9038-4595-973b-8abe1f0655c4 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:48:23.058098 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d675f4-9038-4595-973b-8abe1f0655c4 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:48:23.062793 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52a10eb-5cb4-4c8a-b4c5-85e6544373e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:48:23.063467 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52a10eb-5cb4-4c8a-b4c5-85e6544373e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:48:23.063755 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52a10eb-5cb4-4c8a-b4c5-85e6544373e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:48:23.064037 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52a10eb-5cb4-4c8a-b4c5-85e6544373e3 admin admin] PATH_INFO: `/v3/projects/3cdf4cec62d44ef38075dbfa31f97f63/users/51a9f381b5a249e08d00bcb78b8cf882/roles/759f389b221049c4b01faf3b984cd8d7` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:48:23.079949 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52a10eb-5cb4-4c8a-b4c5-85e6544373e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3cdf4cec62d44ef38075dbfa31f97f63, user_id=51a9f381b5a249e08d00bcb78b8cf882, role_id=759f389b221049c4b01faf3b984cd8d7)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:48:23.082768 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52a10eb-5cb4-4c8a-b4c5-85e6544373e3 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:48:23.097964 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1109/2209] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 13:48:23 2022] PUT /identity/v3/projects/3cdf4cec62d44ef38075dbfa31f97f63/users/51a9f381b5a249e08d00bcb78b8cf882/roles/759f389b221049c4b01faf3b984cd8d7 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:48:23.109227 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df2edfb-380f-42b1-b685-c84320c94bec None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:48:23.109622 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df2edfb-380f-42b1-b685-c84320c94bec None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:48:23.109971 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df2edfb-380f-42b1-b685-c84320c94bec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:48:23.155448 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-6df2edfb-380f-42b1-b685-c84320c94bec None None] MFA Rules not processed for user `51a9f381b5a249e08d00bcb78b8cf882`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:48:23.156678 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-6df2edfb-380f-42b1-b685-c84320c94bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:48:23.234089 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1101/2210] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:48:23 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:48:23.245336 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52a10eb-5cb4-4c8a-b4c5-85e6544373e3 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:48:23.248780 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-a52a10eb-5cb4-4c8a-b4c5-85e6544373e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:48:23.348010 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52a10eb-5cb4-4c8a-b4c5-85e6544373e3 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:48:23.351755 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c815e2-450d-4992-b910-7019c83a6f7e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:48:23.352377 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c815e2-450d-4992-b910-7019c83a6f7e service cinder] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:48:23.352690 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c815e2-450d-4992-b910-7019c83a6f7e service cinder] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:48:23.352969 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c815e2-450d-4992-b910-7019c83a6f7e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:48:23.362617 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c815e2-450d-4992-b910-7019c83a6f7e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:48:23.364209 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c815e2-450d-4992-b910-7019c83a6f7e service cinder] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:48:23.374053 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1110/2211] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:48:23 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:48:25.335457 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df2edfb-380f-42b1-b685-c84320c94bec None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:48:25.336853 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-6df2edfb-380f-42b1-b685-c84320c94bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:48:25.400392 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df2edfb-380f-42b1-b685-c84320c94bec None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:48:25.405154 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607a61e6-94a6-438b-96d4-613a32d2f7f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:48:25.405821 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607a61e6-94a6-438b-96d4-613a32d2f7f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:48:25.406108 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607a61e6-94a6-438b-96d4-613a32d2f7f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:48:25.406383 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607a61e6-94a6-438b-96d4-613a32d2f7f1 admin admin] PATH_INFO: `/v3/users/ea1cd5eb9b0d42e1a94344dcf3e0f180` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:48:25.407565 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607a61e6-94a6-438b-96d4-613a32d2f7f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea1cd5eb9b0d42e1a94344dcf3e0f180)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:48:25.409495 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607a61e6-94a6-438b-96d4-613a32d2f7f1 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:48:25.455311 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-607a61e6-94a6-438b-96d4-613a32d2f7f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea1cd5eb9b0d42e1a94344dcf3e0f180'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:48:25.455950 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-607a61e6-94a6-438b-96d4-613a32d2f7f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea1cd5eb9b0d42e1a94344dcf3e0f180'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:48:25.461181 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:48:25.463528 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-607a61e6-94a6-438b-96d4-613a32d2f7f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea1cd5eb9b0d42e1a94344dcf3e0f180'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:48:25.464761 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-607a61e6-94a6-438b-96d4-613a32d2f7f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea1cd5eb9b0d42e1a94344dcf3e0f180'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:48:25.477766 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-607a61e6-94a6-438b-96d4-613a32d2f7f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea1cd5eb9b0d42e1a94344dcf3e0f180'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:48:25.490790 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1102/2212] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 13:48:25 2022] DELETE /identity/v3/users/ea1cd5eb9b0d42e1a94344dcf3e0f180 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:48:25.495209 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f881d209-8163-4fdf-887d-adc403442c01 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:48:25.495707 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f881d209-8163-4fdf-887d-adc403442c01 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:48:25.495707 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f881d209-8163-4fdf-887d-adc403442c01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:48:25.514872 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-f881d209-8163-4fdf-887d-adc403442c01 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:48:25.516089 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-f881d209-8163-4fdf-887d-adc403442c01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:48:25.577698 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1111/2213] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:48:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:48:25.587300 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607a61e6-94a6-438b-96d4-613a32d2f7f1 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:48:25.588911 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-607a61e6-94a6-438b-96d4-613a32d2f7f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:48:25.661993 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607a61e6-94a6-438b-96d4-613a32d2f7f1 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:48:25.673261 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1b3e4f-9010-4a39-8e76-f2f1862df7a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:48:25.674008 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b3e4f-9010-4a39-8e76-f2f1862df7a9 service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:48:25.674291 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b3e4f-9010-4a39-8e76-f2f1862df7a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:48:25.674569 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b3e4f-9010-4a39-8e76-f2f1862df7a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:48:25.684237 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1b3e4f-9010-4a39-8e76-f2f1862df7a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:48:25.686387 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1b3e4f-9010-4a39-8e76-f2f1862df7a9 service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:48:25.696091 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1103/2214] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:48:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:48:25.969059 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f881d209-8163-4fdf-887d-adc403442c01 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:48:25.970649 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-f881d209-8163-4fdf-887d-adc403442c01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:48:25.983838 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f881d209-8163-4fdf-887d-adc403442c01 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:48:25.989294 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd87e91-16b6-47a3-90a3-0bf2e125d4fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:48:25.990025 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd87e91-16b6-47a3-90a3-0bf2e125d4fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:48:25.990309 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd87e91-16b6-47a3-90a3-0bf2e125d4fa admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:48:25.990585 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd87e91-16b6-47a3-90a3-0bf2e125d4fa admin admin] PATH_INFO: `/v3/projects/d6649531b6f24f7f94c1a1740941f710` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:48:25.992352 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd87e91-16b6-47a3-90a3-0bf2e125d4fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6649531b6f24f7f94c1a1740941f710)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:48:25.994625 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd87e91-16b6-47a3-90a3-0bf2e125d4fa admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:48:26.043514 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-ebd87e91-16b6-47a3-90a3-0bf2e125d4fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6649531b6f24f7f94c1a1740941f710', 'request_id': 'req-ebd87e91-16b6-47a3-90a3-0bf2e125d4fa'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:48:26.051891 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-ebd87e91-16b6-47a3-90a3-0bf2e125d4fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6649531b6f24f7f94c1a1740941f710', 'request_id': 'req-ebd87e91-16b6-47a3-90a3-0bf2e125d4fa'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:48:26.057445 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-ebd87e91-16b6-47a3-90a3-0bf2e125d4fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6649531b6f24f7f94c1a1740941f710', 'request_id': 'req-ebd87e91-16b6-47a3-90a3-0bf2e125d4fa'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:48:26.068246 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-ebd87e91-16b6-47a3-90a3-0bf2e125d4fa admin admin] The token cache is being invalidate because project d6649531b6f24f7f94c1a1740941f710 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86933) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 13:48:26.068532 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-ebd87e91-16b6-47a3-90a3-0bf2e125d4fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:48:26.069154 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-ebd87e91-16b6-47a3-90a3-0bf2e125d4fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:48:26.070678 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1112/2215] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 13:48:25 2022] DELETE /identity/v3/projects/d6649531b6f24f7f94c1a1740941f710 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:48:26.074655 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1b3e4f-9010-4a39-8e76-f2f1862df7a9 service neutron] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:48:26.084275 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1b3e4f-9010-4a39-8e76-f2f1862df7a9 service neutron] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:48:26.143746 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69bdb60-d543-4de0-a8a6-ab0bb2a24182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:48:26.144384 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69bdb60-d543-4de0-a8a6-ab0bb2a24182 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:48:26.144660 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69bdb60-d543-4de0-a8a6-ab0bb2a24182 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:48:26.144931 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69bdb60-d543-4de0-a8a6-ab0bb2a24182 admin admin] PATH_INFO: `/v3/users/c8f96cdc262c4638a66604b88cbb77b3` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:48:26.146096 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69bdb60-d543-4de0-a8a6-ab0bb2a24182 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8f96cdc262c4638a66604b88cbb77b3)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:48:26.147995 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69bdb60-d543-4de0-a8a6-ab0bb2a24182 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:48:26.196111 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-e69bdb60-d543-4de0-a8a6-ab0bb2a24182 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8f96cdc262c4638a66604b88cbb77b3'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:48:26.196781 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-e69bdb60-d543-4de0-a8a6-ab0bb2a24182 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8f96cdc262c4638a66604b88cbb77b3'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:48:26.204115 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-e69bdb60-d543-4de0-a8a6-ab0bb2a24182 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8f96cdc262c4638a66604b88cbb77b3'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:48:26.204764 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-e69bdb60-d543-4de0-a8a6-ab0bb2a24182 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8f96cdc262c4638a66604b88cbb77b3'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:48:26.216993 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-e69bdb60-d543-4de0-a8a6-ab0bb2a24182 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8f96cdc262c4638a66604b88cbb77b3'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:48:26.219600 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:48:26.230302 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1104/2216] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 13:48:26 2022] DELETE /identity/v3/users/c8f96cdc262c4638a66604b88cbb77b3 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:48:26.499119 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd87e91-16b6-47a3-90a3-0bf2e125d4fa admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:48:26.500940 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-ebd87e91-16b6-47a3-90a3-0bf2e125d4fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:48:26.510096 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:48:26.517029 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd87e91-16b6-47a3-90a3-0bf2e125d4fa admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:48:26.576488 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613e261b-7dd8-49ef-8c3e-b08e518b71ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:48:26.577133 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613e261b-7dd8-49ef-8c3e-b08e518b71ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:48:26.577413 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613e261b-7dd8-49ef-8c3e-b08e518b71ad admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:48:26.577717 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613e261b-7dd8-49ef-8c3e-b08e518b71ad admin admin] PATH_INFO: `/v3/projects/fd90c908924b4490a1bc3273f5832cbc` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:48:26.578868 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613e261b-7dd8-49ef-8c3e-b08e518b71ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd90c908924b4490a1bc3273f5832cbc)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:48:26.580794 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613e261b-7dd8-49ef-8c3e-b08e518b71ad admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:48:26.625832 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-613e261b-7dd8-49ef-8c3e-b08e518b71ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd90c908924b4490a1bc3273f5832cbc', 'request_id': 'req-613e261b-7dd8-49ef-8c3e-b08e518b71ad'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:48:26.632824 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-613e261b-7dd8-49ef-8c3e-b08e518b71ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd90c908924b4490a1bc3273f5832cbc', 'request_id': 'req-613e261b-7dd8-49ef-8c3e-b08e518b71ad'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:48:26.638954 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-613e261b-7dd8-49ef-8c3e-b08e518b71ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd90c908924b4490a1bc3273f5832cbc', 'request_id': 'req-613e261b-7dd8-49ef-8c3e-b08e518b71ad'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:48:26.649645 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-613e261b-7dd8-49ef-8c3e-b08e518b71ad admin admin] The token cache is being invalidate because project fd90c908924b4490a1bc3273f5832cbc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86933) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 13:48:26.649930 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-613e261b-7dd8-49ef-8c3e-b08e518b71ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:48:26.650554 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-613e261b-7dd8-49ef-8c3e-b08e518b71ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:48:26.652101 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1113/2217] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 13:48:26 2022] DELETE /identity/v3/projects/fd90c908924b4490a1bc3273f5832cbc => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:48:26.656390 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69bdb60-d543-4de0-a8a6-ab0bb2a24182 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:48:26.658895 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-e69bdb60-d543-4de0-a8a6-ab0bb2a24182 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:48:26.671712 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69bdb60-d543-4de0-a8a6-ab0bb2a24182 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:48:26.734351 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc9a2e8-d7c5-4326-bf1e-d64355d81842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:48:26.735036 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc9a2e8-d7c5-4326-bf1e-d64355d81842 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:48:26.735326 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc9a2e8-d7c5-4326-bf1e-d64355d81842 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:48:26.736184 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc9a2e8-d7c5-4326-bf1e-d64355d81842 admin admin] PATH_INFO: `/v3/users/0384fa2d1ffd4468bb74cb3a5d8c8df3` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:48:26.736806 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc9a2e8-d7c5-4326-bf1e-d64355d81842 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0384fa2d1ffd4468bb74cb3a5d8c8df3)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:48:26.738764 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc9a2e8-d7c5-4326-bf1e-d64355d81842 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:48:26.797808 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-7fc9a2e8-d7c5-4326-bf1e-d64355d81842 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0384fa2d1ffd4468bb74cb3a5d8c8df3'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:48:26.798824 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-7fc9a2e8-d7c5-4326-bf1e-d64355d81842 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0384fa2d1ffd4468bb74cb3a5d8c8df3'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:48:26.809673 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-7fc9a2e8-d7c5-4326-bf1e-d64355d81842 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0384fa2d1ffd4468bb74cb3a5d8c8df3'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:48:26.811783 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-7fc9a2e8-d7c5-4326-bf1e-d64355d81842 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0384fa2d1ffd4468bb74cb3a5d8c8df3'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:48:26.838530 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-7fc9a2e8-d7c5-4326-bf1e-d64355d81842 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0384fa2d1ffd4468bb74cb3a5d8c8df3'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:48:26.867668 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1105/2218] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 13:48:26 2022] DELETE /identity/v3/users/0384fa2d1ffd4468bb74cb3a5d8c8df3 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:48:27.181404 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613e261b-7dd8-49ef-8c3e-b08e518b71ad admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:48:27.182882 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-613e261b-7dd8-49ef-8c3e-b08e518b71ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:48:27.195938 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613e261b-7dd8-49ef-8c3e-b08e518b71ad admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:48:27.253766 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd0ed1a-192d-43ca-a049-5f9822e13e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:48:27.254411 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd0ed1a-192d-43ca-a049-5f9822e13e7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:48:27.254690 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd0ed1a-192d-43ca-a049-5f9822e13e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:48:27.254963 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd0ed1a-192d-43ca-a049-5f9822e13e7c admin admin] PATH_INFO: `/v3/projects/9c7da2007e014012847422658cee1fb8` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:48:27.256188 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd0ed1a-192d-43ca-a049-5f9822e13e7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c7da2007e014012847422658cee1fb8)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:48:27.258122 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd0ed1a-192d-43ca-a049-5f9822e13e7c admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:48:27.297107 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:48:27.308141 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-dcd0ed1a-192d-43ca-a049-5f9822e13e7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c7da2007e014012847422658cee1fb8', 'request_id': 'req-dcd0ed1a-192d-43ca-a049-5f9822e13e7c'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:48:27.315063 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-dcd0ed1a-192d-43ca-a049-5f9822e13e7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c7da2007e014012847422658cee1fb8', 'request_id': 'req-dcd0ed1a-192d-43ca-a049-5f9822e13e7c'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:48:27.320612 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-dcd0ed1a-192d-43ca-a049-5f9822e13e7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c7da2007e014012847422658cee1fb8', 'request_id': 'req-dcd0ed1a-192d-43ca-a049-5f9822e13e7c'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:48:27.331703 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-dcd0ed1a-192d-43ca-a049-5f9822e13e7c admin admin] The token cache is being invalidate because project 9c7da2007e014012847422658cee1fb8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86933) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 13:48:27.331996 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-dcd0ed1a-192d-43ca-a049-5f9822e13e7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:48:27.332623 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-dcd0ed1a-192d-43ca-a049-5f9822e13e7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:48:27.334147 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1114/2219] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 13:48:27 2022] DELETE /identity/v3/projects/9c7da2007e014012847422658cee1fb8 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 13:48:27.339018 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6911ed61-ba8a-44ab-93ae-7d09badd512c None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:48:27.339329 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6911ed61-ba8a-44ab-93ae-7d09badd512c None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:48:27.339637 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6911ed61-ba8a-44ab-93ae-7d09badd512c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:48:27.359834 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-6911ed61-ba8a-44ab-93ae-7d09badd512c None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:48:27.361044 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-6911ed61-ba8a-44ab-93ae-7d09badd512c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:48:27.418490 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1106/2220] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:48:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:48:27.426876 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ac9618-c9ed-4ad3-bce2-fa65dab555dc None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:48:27.427166 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ac9618-c9ed-4ad3-bce2-fa65dab555dc None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:48:27.427446 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ac9618-c9ed-4ad3-bce2-fa65dab555dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:48:27.446532 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-90ac9618-c9ed-4ad3-bce2-fa65dab555dc None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:48:27.447798 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-90ac9618-c9ed-4ad3-bce2-fa65dab555dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:48:27.455665 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1115/2221] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:48:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:48:27.459829 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6911ed61-ba8a-44ab-93ae-7d09badd512c None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:48:27.470210 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6911ed61-ba8a-44ab-93ae-7d09badd512c None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:48:27.474880 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a68d3f-6118-4435-b2e7-3b66294cd2f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:48:27.475499 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a68d3f-6118-4435-b2e7-3b66294cd2f0 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:48:27.475811 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a68d3f-6118-4435-b2e7-3b66294cd2f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:48:27.476086 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a68d3f-6118-4435-b2e7-3b66294cd2f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:48:27.476782 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a68d3f-6118-4435-b2e7-3b66294cd2f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 13:48:27.477092 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a68d3f-6118-4435-b2e7-3b66294cd2f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:48:27.478748 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a68d3f-6118-4435-b2e7-3b66294cd2f0 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:48:27.488311 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:48:27.492035 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1107/2222] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Jul 26 13:48:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 13:48:27.496177 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ac9618-c9ed-4ad3-bce2-fa65dab555dc None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:48:27.505490 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ac9618-c9ed-4ad3-bce2-fa65dab555dc None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:48:27.510094 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8796b503-6537-4ec7-98e9-867d69108d9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:48:27.510688 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8796b503-6537-4ec7-98e9-867d69108d9f admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:48:27.510968 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8796b503-6537-4ec7-98e9-867d69108d9f admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:48:27.511249 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8796b503-6537-4ec7-98e9-867d69108d9f admin admin] PATH_INFO: `/v3/projects` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:48:27.511898 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8796b503-6537-4ec7-98e9-867d69108d9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:48:27.513845 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8796b503-6537-4ec7-98e9-867d69108d9f admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:48:27.540792 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1116/2223] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 13:48:27 2022] POST /identity/v3/projects => generated 371 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:48:27.545454 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a68d3f-6118-4435-b2e7-3b66294cd2f0 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:48:27.555262 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a68d3f-6118-4435-b2e7-3b66294cd2f0 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:48:27.614419 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a20774d-0a83-4e18-8ca3-6c5e3718decd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:48:27.614545 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a20774d-0a83-4e18-8ca3-6c5e3718decd admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:48:27.614759 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a20774d-0a83-4e18-8ca3-6c5e3718decd admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:48:27.615035 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a20774d-0a83-4e18-8ca3-6c5e3718decd admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:48:27.615717 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a20774d-0a83-4e18-8ca3-6c5e3718decd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:48:27.617668 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a20774d-0a83-4e18-8ca3-6c5e3718decd admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:48:27.638689 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1108/2224] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 13:48:27 2022] POST /identity/v3/users => generated 286 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 13:48:27.643000 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8796b503-6537-4ec7-98e9-867d69108d9f admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:48:27.654144 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8796b503-6537-4ec7-98e9-867d69108d9f admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:48:27.658845 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6512e574-6470-4e46-8578-87b5fd6ade57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:48:27.659455 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6512e574-6470-4e46-8578-87b5fd6ade57 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:48:27.659739 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6512e574-6470-4e46-8578-87b5fd6ade57 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:48:27.660015 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6512e574-6470-4e46-8578-87b5fd6ade57 admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:48:27.660706 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6512e574-6470-4e46-8578-87b5fd6ade57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:48:27.662351 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6512e574-6470-4e46-8578-87b5fd6ade57 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:48:27.672147 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1117/2225] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:48:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:48:27.675960 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a20774d-0a83-4e18-8ca3-6c5e3718decd admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:48:27.686369 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a20774d-0a83-4e18-8ca3-6c5e3718decd admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:48:27.691000 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7394755e-a26a-4ea2-8c48-38c0046ac566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:48:27.691612 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7394755e-a26a-4ea2-8c48-38c0046ac566 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:48:27.691896 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7394755e-a26a-4ea2-8c48-38c0046ac566 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:48:27.692175 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7394755e-a26a-4ea2-8c48-38c0046ac566 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:48:27.692869 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7394755e-a26a-4ea2-8c48-38c0046ac566 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:48:27.694511 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7394755e-a26a-4ea2-8c48-38c0046ac566 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:48:27.705746 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1109/2226] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 13:48:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 13:48:27.709999 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6512e574-6470-4e46-8578-87b5fd6ade57 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:48:27.720587 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6512e574-6470-4e46-8578-87b5fd6ade57 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:48:27.725569 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9961ff38-8460-4b7d-87b8-b65dd6042972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:48:27.726231 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9961ff38-8460-4b7d-87b8-b65dd6042972 admin admin] REQUEST_METHOD: `PUT` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:48:27.726519 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9961ff38-8460-4b7d-87b8-b65dd6042972 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:48:27.726801 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9961ff38-8460-4b7d-87b8-b65dd6042972 admin admin] PATH_INFO: `/v3/projects/7152a4737f0c488ea30329c9876f8739/users/deff226622a845b498ff2dd9a18cc4d8/roles/759f389b221049c4b01faf3b984cd8d7` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:48:27.741496 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9961ff38-8460-4b7d-87b8-b65dd6042972 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7152a4737f0c488ea30329c9876f8739, user_id=deff226622a845b498ff2dd9a18cc4d8, role_id=759f389b221049c4b01faf3b984cd8d7)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:48:27.744118 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9961ff38-8460-4b7d-87b8-b65dd6042972 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:48:27.765096 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1118/2227] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 13:48:27 2022] PUT /identity/v3/projects/7152a4737f0c488ea30329c9876f8739/users/deff226622a845b498ff2dd9a18cc4d8/roles/759f389b221049c4b01faf3b984cd8d7 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:48:27.839302 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ca66d4-3cf3-4cde-945c-b9bd5fc00151 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:48:27.839591 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ca66d4-3cf3-4cde-945c-b9bd5fc00151 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:48:27.839898 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ca66d4-3cf3-4cde-945c-b9bd5fc00151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:48:27.867384 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-c0ca66d4-3cf3-4cde-945c-b9bd5fc00151 None None] MFA Rules not processed for user `deff226622a845b498ff2dd9a18cc4d8`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:48:27.868587 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-c0ca66d4-3cf3-4cde-945c-b9bd5fc00151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:48:27.932031 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1110/2228] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:48:27 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:48:27.943414 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9961ff38-8460-4b7d-87b8-b65dd6042972 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:48:27.945124 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-9961ff38-8460-4b7d-87b8-b65dd6042972 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:48:28.007216 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9961ff38-8460-4b7d-87b8-b65dd6042972 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:48:28.010953 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69519b30-9149-4356-8004-41a95f45bda4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:48:28.011642 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69519b30-9149-4356-8004-41a95f45bda4 service cinder] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:48:28.011928 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69519b30-9149-4356-8004-41a95f45bda4 service cinder] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:48:28.012203 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69519b30-9149-4356-8004-41a95f45bda4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:48:28.022352 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69519b30-9149-4356-8004-41a95f45bda4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:48:28.023779 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69519b30-9149-4356-8004-41a95f45bda4 service cinder] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:48:28.035653 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1119/2229] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:48:27 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:48:28.062916 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ca66d4-3cf3-4cde-945c-b9bd5fc00151 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:48:28.128918 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ca66d4-3cf3-4cde-945c-b9bd5fc00151 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:48:28.133517 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56141606-63c2-4520-91d7-d3e950314201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:48:28.134175 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56141606-63c2-4520-91d7-d3e950314201 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:48:28.134454 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56141606-63c2-4520-91d7-d3e950314201 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:48:28.134728 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56141606-63c2-4520-91d7-d3e950314201 admin admin] PATH_INFO: `/v3/users/deff226622a845b498ff2dd9a18cc4d8` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:48:28.135946 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56141606-63c2-4520-91d7-d3e950314201 admin admin] RBAC: Authorizing `identity:delete_user(user_id=deff226622a845b498ff2dd9a18cc4d8)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:48:28.138477 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56141606-63c2-4520-91d7-d3e950314201 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:48:28.150034 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:48:28.197577 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-56141606-63c2-4520-91d7-d3e950314201 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'deff226622a845b498ff2dd9a18cc4d8'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:48:28.198186 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-56141606-63c2-4520-91d7-d3e950314201 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'deff226622a845b498ff2dd9a18cc4d8'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:48:28.206076 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-56141606-63c2-4520-91d7-d3e950314201 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'deff226622a845b498ff2dd9a18cc4d8'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:48:28.206808 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-56141606-63c2-4520-91d7-d3e950314201 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'deff226622a845b498ff2dd9a18cc4d8'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:48:28.219061 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-56141606-63c2-4520-91d7-d3e950314201 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'deff226622a845b498ff2dd9a18cc4d8'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:48:28.232993 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1111/2230] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 13:48:28 2022] DELETE /identity/v3/users/deff226622a845b498ff2dd9a18cc4d8 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:48:28.237789 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e581cea-fc6d-48a4-bda2-f04cbf0e2676 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:48:28.238083 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e581cea-fc6d-48a4-bda2-f04cbf0e2676 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:48:28.238366 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e581cea-fc6d-48a4-bda2-f04cbf0e2676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:48:28.258755 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-8e581cea-fc6d-48a4-bda2-f04cbf0e2676 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:48:28.259999 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-8e581cea-fc6d-48a4-bda2-f04cbf0e2676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:48:28.324144 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1120/2231] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:48:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:48:28.335211 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56141606-63c2-4520-91d7-d3e950314201 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:48:28.336813 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-56141606-63c2-4520-91d7-d3e950314201 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:48:28.397859 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56141606-63c2-4520-91d7-d3e950314201 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:48:28.401778 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baf3476-b575-4df1-9801-3636c9334fd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:48:28.402786 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baf3476-b575-4df1-9801-3636c9334fd1 service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:48:28.403422 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baf3476-b575-4df1-9801-3636c9334fd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:48:28.403825 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baf3476-b575-4df1-9801-3636c9334fd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:48:28.414098 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baf3476-b575-4df1-9801-3636c9334fd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:48:28.426244 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baf3476-b575-4df1-9801-3636c9334fd1 service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:48:28.426244 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1112/2232] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:48:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:48:28.690741 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e581cea-fc6d-48a4-bda2-f04cbf0e2676 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:48:28.692309 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-8e581cea-fc6d-48a4-bda2-f04cbf0e2676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:48:28.705301 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:48:28.712163 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e581cea-fc6d-48a4-bda2-f04cbf0e2676 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:48:28.718571 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae10b30e-2855-40bc-a5ae-19c065e905eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:48:28.720839 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae10b30e-2855-40bc-a5ae-19c065e905eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:48:28.721529 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae10b30e-2855-40bc-a5ae-19c065e905eb admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:48:28.721892 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae10b30e-2855-40bc-a5ae-19c065e905eb admin admin] PATH_INFO: `/v3/projects/7152a4737f0c488ea30329c9876f8739` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:48:28.723640 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae10b30e-2855-40bc-a5ae-19c065e905eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=7152a4737f0c488ea30329c9876f8739)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:48:28.734723 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae10b30e-2855-40bc-a5ae-19c065e905eb admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:48:28.789885 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-ae10b30e-2855-40bc-a5ae-19c065e905eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7152a4737f0c488ea30329c9876f8739', 'request_id': 'req-ae10b30e-2855-40bc-a5ae-19c065e905eb'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:48:28.799561 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-ae10b30e-2855-40bc-a5ae-19c065e905eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7152a4737f0c488ea30329c9876f8739', 'request_id': 'req-ae10b30e-2855-40bc-a5ae-19c065e905eb'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:48:28.807973 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-ae10b30e-2855-40bc-a5ae-19c065e905eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7152a4737f0c488ea30329c9876f8739', 'request_id': 'req-ae10b30e-2855-40bc-a5ae-19c065e905eb'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:48:28.821112 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-ae10b30e-2855-40bc-a5ae-19c065e905eb admin admin] The token cache is being invalidate because project 7152a4737f0c488ea30329c9876f8739 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86933) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 13:48:28.821409 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-ae10b30e-2855-40bc-a5ae-19c065e905eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:48:28.822392 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-ae10b30e-2855-40bc-a5ae-19c065e905eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:48:28.824182 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1121/2233] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 13:48:28 2022] DELETE /identity/v3/projects/7152a4737f0c488ea30329c9876f8739 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:48:31.548690 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baf3476-b575-4df1-9801-3636c9334fd1 service neutron] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:48:31.550136 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-9baf3476-b575-4df1-9801-3636c9334fd1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:48:31.563470 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baf3476-b575-4df1-9801-3636c9334fd1 service neutron] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:48:31.620007 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22712b2-44b4-4843-94e0-47daef54c1bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:48:31.620664 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22712b2-44b4-4843-94e0-47daef54c1bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:48:31.620944 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22712b2-44b4-4843-94e0-47daef54c1bc admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:48:31.621217 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22712b2-44b4-4843-94e0-47daef54c1bc admin admin] PATH_INFO: `/v3/users/51a9f381b5a249e08d00bcb78b8cf882` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:48:31.622945 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22712b2-44b4-4843-94e0-47daef54c1bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=51a9f381b5a249e08d00bcb78b8cf882)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:48:31.624872 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22712b2-44b4-4843-94e0-47daef54c1bc admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:48:31.657313 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:48:31.671495 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-a22712b2-44b4-4843-94e0-47daef54c1bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51a9f381b5a249e08d00bcb78b8cf882'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:48:31.672136 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-a22712b2-44b4-4843-94e0-47daef54c1bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51a9f381b5a249e08d00bcb78b8cf882'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:48:31.679226 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-a22712b2-44b4-4843-94e0-47daef54c1bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51a9f381b5a249e08d00bcb78b8cf882'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:48:31.679865 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-a22712b2-44b4-4843-94e0-47daef54c1bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51a9f381b5a249e08d00bcb78b8cf882'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:48:31.692121 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-a22712b2-44b4-4843-94e0-47daef54c1bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51a9f381b5a249e08d00bcb78b8cf882'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:48:31.704426 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1113/2234] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 13:48:31 2022] DELETE /identity/v3/users/51a9f381b5a249e08d00bcb78b8cf882 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:48:31.708791 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91c0d76-34c6-4e66-9db1-cc2a91480df8 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:48:31.709254 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91c0d76-34c6-4e66-9db1-cc2a91480df8 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:48:31.709356 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91c0d76-34c6-4e66-9db1-cc2a91480df8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:48:31.728319 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-d91c0d76-34c6-4e66-9db1-cc2a91480df8 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:48:31.729818 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-d91c0d76-34c6-4e66-9db1-cc2a91480df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:48:31.790933 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1122/2235] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:48:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:48:31.802204 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22712b2-44b4-4843-94e0-47daef54c1bc admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:48:31.803987 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-a22712b2-44b4-4843-94e0-47daef54c1bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:48:31.866945 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22712b2-44b4-4843-94e0-47daef54c1bc admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:48:31.871055 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342d3a2f-9235-45cc-83ae-79ba1fa617a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:48:31.871927 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342d3a2f-9235-45cc-83ae-79ba1fa617a9 service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:48:31.872218 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342d3a2f-9235-45cc-83ae-79ba1fa617a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:48:31.872567 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342d3a2f-9235-45cc-83ae-79ba1fa617a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:48:31.882583 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342d3a2f-9235-45cc-83ae-79ba1fa617a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:48:31.884018 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342d3a2f-9235-45cc-83ae-79ba1fa617a9 service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:48:31.893797 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1114/2236] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:48:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:48:32.167152 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91c0d76-34c6-4e66-9db1-cc2a91480df8 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:48:32.168676 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-d91c0d76-34c6-4e66-9db1-cc2a91480df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:48:32.181505 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91c0d76-34c6-4e66-9db1-cc2a91480df8 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:48:32.186582 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18622f37-a0b5-44b7-acdf-ad4d520cec64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:48:32.188507 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18622f37-a0b5-44b7-acdf-ad4d520cec64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:48:32.188794 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18622f37-a0b5-44b7-acdf-ad4d520cec64 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:48:32.189066 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18622f37-a0b5-44b7-acdf-ad4d520cec64 admin admin] PATH_INFO: `/v3/projects/3cdf4cec62d44ef38075dbfa31f97f63` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:48:32.190552 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18622f37-a0b5-44b7-acdf-ad4d520cec64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3cdf4cec62d44ef38075dbfa31f97f63)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:48:32.192516 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18622f37-a0b5-44b7-acdf-ad4d520cec64 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:48:32.238787 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-18622f37-a0b5-44b7-acdf-ad4d520cec64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3cdf4cec62d44ef38075dbfa31f97f63', 'request_id': 'req-18622f37-a0b5-44b7-acdf-ad4d520cec64'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:48:32.245917 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-18622f37-a0b5-44b7-acdf-ad4d520cec64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3cdf4cec62d44ef38075dbfa31f97f63', 'request_id': 'req-18622f37-a0b5-44b7-acdf-ad4d520cec64'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:48:32.251918 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-18622f37-a0b5-44b7-acdf-ad4d520cec64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3cdf4cec62d44ef38075dbfa31f97f63', 'request_id': 'req-18622f37-a0b5-44b7-acdf-ad4d520cec64'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:48:32.263654 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-18622f37-a0b5-44b7-acdf-ad4d520cec64 admin admin] The token cache is being invalidate because project 3cdf4cec62d44ef38075dbfa31f97f63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86933) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 13:48:32.263940 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-18622f37-a0b5-44b7-acdf-ad4d520cec64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:48:32.264584 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-18622f37-a0b5-44b7-acdf-ad4d520cec64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 13:48:32.266136 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1123/2237] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 13:48:32 2022] DELETE /identity/v3/projects/3cdf4cec62d44ef38075dbfa31f97f63 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 13:48:41.889422 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=43,keystone:DELETE=5,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:48:42.260276 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=80,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:48:55.095506 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f781a537-e8c4-4e5d-bad8-59261a95a5e1 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:48:55.096017 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f781a537-e8c4-4e5d-bad8-59261a95a5e1 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:48:55.096651 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f781a537-e8c4-4e5d-bad8-59261a95a5e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:48:55.119923 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-f781a537-e8c4-4e5d-bad8-59261a95a5e1 None None] MFA Rules not processed for user `56db6603f359477885b5151dec71c096`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:48:55.121215 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-f781a537-e8c4-4e5d-bad8-59261a95a5e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:48:55.184120 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1115/2238] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:48:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:49:04.202606 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcc6aad-c10c-42fe-94a4-f5d848350f00 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:49:04.203048 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcc6aad-c10c-42fe-94a4-f5d848350f00 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:49:04.203141 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcc6aad-c10c-42fe-94a4-f5d848350f00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:49:04.222223 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-3bcc6aad-c10c-42fe-94a4-f5d848350f00 None None] MFA Rules not processed for user `0accc3551cd348e2a946bde38e4db40f`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:49:04.223537 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-3bcc6aad-c10c-42fe-94a4-f5d848350f00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:49:04.277849 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1124/2239] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:49:04 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:49:05.176228 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=25 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:49:12.919962 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b07ae7-deaf-4b9f-966b-2ae85e39f824 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:49:12.920549 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b07ae7-deaf-4b9f-966b-2ae85e39f824 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:49:12.920549 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b07ae7-deaf-4b9f-966b-2ae85e39f824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:49:12.943510 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-25b07ae7-deaf-4b9f-966b-2ae85e39f824 None None] MFA Rules not processed for user `b64bd7b183d24d8c9037b261c7884810`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:49:12.944749 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-25b07ae7-deaf-4b9f-966b-2ae85e39f824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:49:13.007551 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1116/2240] 10.222.0.58 () {58 vars in 1009 bytes} [Tue Jul 26 13:49:12 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:49:13.052283 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcc6aad-c10c-42fe-94a4-f5d848350f00 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:49:13.053766 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-3bcc6aad-c10c-42fe-94a4-f5d848350f00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:49:13.122430 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcc6aad-c10c-42fe-94a4-f5d848350f00 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:49:13.127387 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93b50b4-0a7b-488d-aaa0-9c6defdf5351 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab2819bac844163ae4305e4f2186605', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:49:13.128018 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93b50b4-0a7b-488d-aaa0-9c6defdf5351 service swift] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:49:13.128305 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93b50b4-0a7b-488d-aaa0-9c6defdf5351 service swift] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:49:13.128579 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93b50b4-0a7b-488d-aaa0-9c6defdf5351 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:49:13.137927 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93b50b4-0a7b-488d-aaa0-9c6defdf5351 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:49:13.139517 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93b50b4-0a7b-488d-aaa0-9c6defdf5351 service swift] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:49:13.148025 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1125/2241] 10.222.0.58 () {60 vars in 1384 bytes} [Tue Jul 26 13:49:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 13:49:13.443538 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd86c28-8f93-418b-bcd5-e160cff2ab18 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:49:13.444335 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd86c28-8f93-418b-bcd5-e160cff2ab18 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:49:13.444824 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd86c28-8f93-418b-bcd5-e160cff2ab18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:49:13.467354 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-dcd86c28-8f93-418b-bcd5-e160cff2ab18 None None] MFA Rules not processed for user `22b8c6b94b8741b085081ca02e5316e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:49:13.469158 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-dcd86c28-8f93-418b-bcd5-e160cff2ab18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:49:13.545622 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1117/2242] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 13:49:13 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:49:23.144962 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=57 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:49:23.527001 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=50 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:49:28.404851 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93b50b4-0a7b-488d-aaa0-9c6defdf5351 service swift] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:49:28.406385 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-b93b50b4-0a7b-488d-aaa0-9c6defdf5351 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:49:28.473583 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93b50b4-0a7b-488d-aaa0-9c6defdf5351 service swift] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:49:28.478463 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b324c36f-801b-43e2-8fe7-0cddd7142493 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '835cbc13756b41b598142df3e35f5492', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:49:28.479570 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b324c36f-801b-43e2-8fe7-0cddd7142493 service placement] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:49:28.479846 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b324c36f-801b-43e2-8fe7-0cddd7142493 service placement] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:49:28.480116 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b324c36f-801b-43e2-8fe7-0cddd7142493 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:49:28.481850 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-b324c36f-801b-43e2-8fe7-0cddd7142493 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:49:28.492739 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b324c36f-801b-43e2-8fe7-0cddd7142493 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:49:28.494339 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b324c36f-801b-43e2-8fe7-0cddd7142493 service placement] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:49:28.504533 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1126/2243] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:49:28 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:49:30.417560 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d717a8cf-4f80-42a9-ad8f-1f9073c76015 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:49:30.418127 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d717a8cf-4f80-42a9-ad8f-1f9073c76015 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:49:30.418127 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d717a8cf-4f80-42a9-ad8f-1f9073c76015 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:49:30.419421 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1118/2244] 10.222.0.58 () {54 vars in 921 bytes} [Tue Jul 26 13:49:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:49:30.425731 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eccbdd4-1464-4b2b-9d35-cfb92ca0aa25 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:49:30.426198 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eccbdd4-1464-4b2b-9d35-cfb92ca0aa25 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:49:30.426292 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eccbdd4-1464-4b2b-9d35-cfb92ca0aa25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:49:30.445731 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-3eccbdd4-1464-4b2b-9d35-cfb92ca0aa25 None None] MFA Rules not processed for user `c69161cd520b4b29bf50f3bf2f1c033b`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:49:30.446964 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-3eccbdd4-1464-4b2b-9d35-cfb92ca0aa25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:49:30.515102 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1127/2245] 10.222.0.58 () {58 vars in 1018 bytes} [Tue Jul 26 13:49:30 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:49:30.527439 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d717a8cf-4f80-42a9-ad8f-1f9073c76015 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:49:30.529150 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-d717a8cf-4f80-42a9-ad8f-1f9073c76015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:49:30.611593 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d717a8cf-4f80-42a9-ad8f-1f9073c76015 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:49:30.615214 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b773a1-2d4a-4cb1-aea5-0b1685bb4461 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf065aef9a046b2adaec34816a336b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:49:30.615839 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b773a1-2d4a-4cb1-aea5-0b1685bb4461 service glance] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:49:30.616118 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b773a1-2d4a-4cb1-aea5-0b1685bb4461 service glance] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:49:30.616387 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b773a1-2d4a-4cb1-aea5-0b1685bb4461 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:49:30.626380 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b773a1-2d4a-4cb1-aea5-0b1685bb4461 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:49:30.627783 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b773a1-2d4a-4cb1-aea5-0b1685bb4461 service glance] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:49:30.638738 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1119/2246] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:49:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 13:49:30.707616 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fe2025-7b7c-46a8-ab6b-4c3904842a73 None None] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:49:30.707901 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fe2025-7b7c-46a8-ab6b-4c3904842a73 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:49:30.708172 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fe2025-7b7c-46a8-ab6b-4c3904842a73 None None] PATH_INFO: `/` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:49:30.709576 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1128/2247] 10.222.0.58 () {54 vars in 921 bytes} [Tue Jul 26 13:49:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 13:49:30.714801 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea11e5e0-530f-4b88-bb91-2d9d0dc4f2ea None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:49:30.715092 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea11e5e0-530f-4b88-bb91-2d9d0dc4f2ea None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:49:30.715373 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea11e5e0-530f-4b88-bb91-2d9d0dc4f2ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:49:30.736444 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-ea11e5e0-530f-4b88-bb91-2d9d0dc4f2ea None None] MFA Rules not processed for user `3223a78866d94c2ba66402911c2dfae9`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:49:30.737678 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-ea11e5e0-530f-4b88-bb91-2d9d0dc4f2ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:49:30.800484 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1120/2248] 10.222.0.58 () {58 vars in 1018 bytes} [Tue Jul 26 13:49:30 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:49:30.812764 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fe2025-7b7c-46a8-ab6b-4c3904842a73 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:49:30.814508 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-b7fe2025-7b7c-46a8-ab6b-4c3904842a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:49:30.827940 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fe2025-7b7c-46a8-ab6b-4c3904842a73 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:49:30.831370 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321e109a-eabe-4642-88b9-d2417d5c17c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:49:30.831996 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321e109a-eabe-4642-88b9-d2417d5c17c4 service neutron] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:49:30.832277 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321e109a-eabe-4642-88b9-d2417d5c17c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:49:30.832550 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321e109a-eabe-4642-88b9-d2417d5c17c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:49:30.840052 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321e109a-eabe-4642-88b9-d2417d5c17c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:49:30.841446 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321e109a-eabe-4642-88b9-d2417d5c17c4 service neutron] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:49:30.849121 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1129/2249] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:49:30 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:49:30.929449 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efabbd27-6bbe-4f73-90bf-3fb909b2f0c1 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:49:30.929775 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efabbd27-6bbe-4f73-90bf-3fb909b2f0c1 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:49:30.930054 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efabbd27-6bbe-4f73-90bf-3fb909b2f0c1 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:49:30.931426 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1121/2250] 10.222.0.58 () {54 vars in 921 bytes} [Tue Jul 26 13:49:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 13:49:30.936179 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35802115-5cd2-4c58-ad95-95970783e0c7 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:49:30.936476 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35802115-5cd2-4c58-ad95-95970783e0c7 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:49:30.936745 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35802115-5cd2-4c58-ad95-95970783e0c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:49:30.956315 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-35802115-5cd2-4c58-ad95-95970783e0c7 None None] MFA Rules not processed for user `c69161cd520b4b29bf50f3bf2f1c033b`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:49:30.957540 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-35802115-5cd2-4c58-ad95-95970783e0c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:49:30.967243 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1130/2251] 10.222.0.58 () {58 vars in 1018 bytes} [Tue Jul 26 13:49:30 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:49:30.977171 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efabbd27-6bbe-4f73-90bf-3fb909b2f0c1 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:49:30.987819 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efabbd27-6bbe-4f73-90bf-3fb909b2f0c1 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:49:30.991336 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad451013-a40f-4cc2-a5ed-3631b191064e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:49:30.991930 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad451013-a40f-4cc2-a5ed-3631b191064e service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:49:30.992209 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad451013-a40f-4cc2-a5ed-3631b191064e service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:49:30.992481 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad451013-a40f-4cc2-a5ed-3631b191064e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:49:31.002803 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad451013-a40f-4cc2-a5ed-3631b191064e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:49:31.004202 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad451013-a40f-4cc2-a5ed-3631b191064e service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:49:31.015118 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1122/2252] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:49:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:49:38.277976 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddc589c-908e-4861-ba89-e3623616f40d None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:49:38.278442 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddc589c-908e-4861-ba89-e3623616f40d None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:49:38.278574 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddc589c-908e-4861-ba89-e3623616f40d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:49:38.297277 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-2ddc589c-908e-4861-ba89-e3623616f40d None None] MFA Rules not processed for user `b64bd7b183d24d8c9037b261c7884810`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:49:38.298499 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-2ddc589c-908e-4861-ba89-e3623616f40d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:49:38.305934 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1131/2253] 10.222.0.58 () {58 vars in 1009 bytes} [Tue Jul 26 13:49:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:49:38.318221 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad451013-a40f-4cc2-a5ed-3631b191064e service neutron] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:49:38.329878 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad451013-a40f-4cc2-a5ed-3631b191064e service neutron] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:49:38.334787 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e55bd0e-81b8-4d48-b1a4-f9bc62f9ff51 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab2819bac844163ae4305e4f2186605', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:49:38.335385 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e55bd0e-81b8-4d48-b1a4-f9bc62f9ff51 service swift] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:49:38.335664 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e55bd0e-81b8-4d48-b1a4-f9bc62f9ff51 service swift] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:49:38.335936 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e55bd0e-81b8-4d48-b1a4-f9bc62f9ff51 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:49:38.344473 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e55bd0e-81b8-4d48-b1a4-f9bc62f9ff51 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:49:38.346132 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e55bd0e-81b8-4d48-b1a4-f9bc62f9ff51 service swift] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:49:38.354052 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1123/2254] 10.222.0.58 () {60 vars in 1384 bytes} [Tue Jul 26 13:49:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 13:49:48.292340 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=75 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:49:48.350838 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=70 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:50:50.114827 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d3afd9-bce1-4238-bb14-eb01b2a5acda None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:50:50.115346 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d3afd9-bce1-4238-bb14-eb01b2a5acda None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:50:50.115454 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d3afd9-bce1-4238-bb14-eb01b2a5acda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:50:50.136158 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-71d3afd9-bce1-4238-bb14-eb01b2a5acda None None] MFA Rules not processed for user `b64bd7b183d24d8c9037b261c7884810`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 13:50:50.137760 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-71d3afd9-bce1-4238-bb14-eb01b2a5acda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:50:50.146201 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1132/2255] 10.222.0.58 () {58 vars in 967 bytes} [Tue Jul 26 13:50:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 13:50:50.189359 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e55bd0e-81b8-4d48-b1a4-f9bc62f9ff51 service swift] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:50:50.200283 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e55bd0e-81b8-4d48-b1a4-f9bc62f9ff51 service swift] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:50:50.205566 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e5afbc-da4b-453b-9bdc-65efa97cd179 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab2819bac844163ae4305e4f2186605', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:50:50.206190 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e5afbc-da4b-453b-9bdc-65efa97cd179 service swift] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:50:50.206489 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e5afbc-da4b-453b-9bdc-65efa97cd179 service swift] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:50:50.206771 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e5afbc-da4b-453b-9bdc-65efa97cd179 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:50:50.215210 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e5afbc-da4b-453b-9bdc-65efa97cd179 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:50:50.216789 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e5afbc-da4b-453b-9bdc-65efa97cd179 service swift] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:50:50.224763 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1124/2256] 10.222.0.58 () {60 vars in 1384 bytes} [Tue Jul 26 13:50:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 13:51:00.129734 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=3 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:51:00.221516 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=6 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:51:30.298355 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d3afd9-bce1-4238-bb14-eb01b2a5acda None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:51:30.299775 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-71d3afd9-bce1-4238-bb14-eb01b2a5acda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:51:30.354182 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d3afd9-bce1-4238-bb14-eb01b2a5acda None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:51:30.357759 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687cd790-e7aa-4f31-af29-0aae61838a14 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:51:30.358382 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687cd790-e7aa-4f31-af29-0aae61838a14 service cinder] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:51:30.358662 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687cd790-e7aa-4f31-af29-0aae61838a14 service cinder] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:51:30.358933 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687cd790-e7aa-4f31-af29-0aae61838a14 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:51:30.361037 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-687cd790-e7aa-4f31-af29-0aae61838a14 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:51:30.375017 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687cd790-e7aa-4f31-af29-0aae61838a14 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:51:30.376432 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687cd790-e7aa-4f31-af29-0aae61838a14 service cinder] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:51:30.387325 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1133/2257] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:51:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:51:32.092917 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e5afbc-da4b-453b-9bdc-65efa97cd179 service swift] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:51:32.102207 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e5afbc-da4b-453b-9bdc-65efa97cd179 service swift] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:51:32.106395 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac018fd-dd6f-431b-a61c-b80664b4f4fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:51:32.106989 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac018fd-dd6f-431b-a61c-b80664b4f4fe service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:51:32.107271 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac018fd-dd6f-431b-a61c-b80664b4f4fe service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:51:32.107544 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac018fd-dd6f-431b-a61c-b80664b4f4fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:51:32.109272 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-1ac018fd-dd6f-431b-a61c-b80664b4f4fe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:51:32.122017 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac018fd-dd6f-431b-a61c-b80664b4f4fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:51:32.123410 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac018fd-dd6f-431b-a61c-b80664b4f4fe service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:51:32.134349 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1125/2258] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:51:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:51:32.622975 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687cd790-e7aa-4f31-af29-0aae61838a14 service cinder] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:51:32.624442 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-687cd790-e7aa-4f31-af29-0aae61838a14 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:51:32.638818 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687cd790-e7aa-4f31-af29-0aae61838a14 service cinder] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:51:32.644233 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4795c3ec-89ac-4fc7-be72-4573d8b2f0d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c69161cd520b4b29bf50f3bf2f1c033b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:51:32.644842 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4795c3ec-89ac-4fc7-be72-4573d8b2f0d6 service nova] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:51:32.645152 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4795c3ec-89ac-4fc7-be72-4573d8b2f0d6 service nova] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:51:32.645427 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4795c3ec-89ac-4fc7-be72-4573d8b2f0d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:51:32.647144 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-4795c3ec-89ac-4fc7-be72-4573d8b2f0d6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:51:32.707656 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4795c3ec-89ac-4fc7-be72-4573d8b2f0d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:51:32.709232 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4795c3ec-89ac-4fc7-be72-4573d8b2f0d6 service nova] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:51:32.718909 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1134/2259] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:51:32 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:51:32.967566 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac018fd-dd6f-431b-a61c-b80664b4f4fe service cinder] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:51:32.969058 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-1ac018fd-dd6f-431b-a61c-b80664b4f4fe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:51:32.983862 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac018fd-dd6f-431b-a61c-b80664b4f4fe service cinder] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:51:32.989402 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babbf450-351e-4f38-a666-a0c477feaf0b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c69161cd520b4b29bf50f3bf2f1c033b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:51:32.990042 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babbf450-351e-4f38-a666-a0c477feaf0b service nova] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:51:32.990325 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babbf450-351e-4f38-a666-a0c477feaf0b service nova] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:51:32.990601 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babbf450-351e-4f38-a666-a0c477feaf0b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:51:32.992394 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-babbf450-351e-4f38-a666-a0c477feaf0b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:51:33.003399 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babbf450-351e-4f38-a666-a0c477feaf0b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:51:33.004964 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babbf450-351e-4f38-a666-a0c477feaf0b service nova] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:51:33.014659 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1126/2260] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:51:32 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:51:37.158907 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4795c3ec-89ac-4fc7-be72-4573d8b2f0d6 service nova] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:51:37.168096 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4795c3ec-89ac-4fc7-be72-4573d8b2f0d6 service nova] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:51:37.171623 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb3e48e-21f2-4be7-bd3a-00c0a420e6f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:51:37.172225 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb3e48e-21f2-4be7-bd3a-00c0a420e6f7 service cinder] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:51:37.172504 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb3e48e-21f2-4be7-bd3a-00c0a420e6f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:51:37.172775 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb3e48e-21f2-4be7-bd3a-00c0a420e6f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:51:37.174535 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-ddb3e48e-21f2-4be7-bd3a-00c0a420e6f7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:51:37.185002 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb3e48e-21f2-4be7-bd3a-00c0a420e6f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:51:37.186370 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb3e48e-21f2-4be7-bd3a-00c0a420e6f7 service cinder] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:51:37.195057 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1135/2261] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:51:37 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:51:43.009821 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=12 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:51:47.191025 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=60 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:52:25.472280 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babbf450-351e-4f38-a666-a0c477feaf0b service nova] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:52:25.481864 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babbf450-351e-4f38-a666-a0c477feaf0b service nova] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:52:25.485363 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c4187d-8d68-4692-b1a6-4f946f51b8d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:52:25.486001 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c4187d-8d68-4692-b1a6-4f946f51b8d7 service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:52:25.486279 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c4187d-8d68-4692-b1a6-4f946f51b8d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:52:25.486548 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c4187d-8d68-4692-b1a6-4f946f51b8d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:52:25.488236 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-43c4187d-8d68-4692-b1a6-4f946f51b8d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:52:25.497392 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c4187d-8d68-4692-b1a6-4f946f51b8d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:52:25.498827 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c4187d-8d68-4692-b1a6-4f946f51b8d7 service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:52:25.506502 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1127/2262] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:52:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:52:35.503127 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=6 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:52:36.404224 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb3e48e-21f2-4be7-bd3a-00c0a420e6f7 service cinder] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:52:36.412708 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb3e48e-21f2-4be7-bd3a-00c0a420e6f7 service cinder] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:52:36.416320 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ae6bac-8c93-40af-9404-63eb954df1ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:52:36.416921 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ae6bac-8c93-40af-9404-63eb954df1ce service cinder] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:52:36.417204 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ae6bac-8c93-40af-9404-63eb954df1ce service cinder] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:52:36.417505 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ae6bac-8c93-40af-9404-63eb954df1ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:52:36.419247 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-63ae6bac-8c93-40af-9404-63eb954df1ce service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:52:36.431118 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ae6bac-8c93-40af-9404-63eb954df1ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:52:36.432570 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ae6bac-8c93-40af-9404-63eb954df1ce service cinder] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:52:36.442198 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1136/2263] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:52:36 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:52:46.437643 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=6 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:53:17.868304 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c4187d-8d68-4692-b1a6-4f946f51b8d7 service neutron] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:53:17.869754 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-43c4187d-8d68-4692-b1a6-4f946f51b8d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:53:17.881181 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c4187d-8d68-4692-b1a6-4f946f51b8d7 service neutron] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:53:17.884738 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c553c4-aa3e-453a-bb13-c65a3fb721bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:53:17.885343 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c553c4-aa3e-453a-bb13-c65a3fb721bd service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:53:17.886190 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c553c4-aa3e-453a-bb13-c65a3fb721bd service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:53:17.886473 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c553c4-aa3e-453a-bb13-c65a3fb721bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:53:17.888187 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-23c553c4-aa3e-453a-bb13-c65a3fb721bd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:53:17.947470 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c553c4-aa3e-453a-bb13-c65a3fb721bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:53:17.949395 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c553c4-aa3e-453a-bb13-c65a3fb721bd service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:53:17.958478 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1128/2264] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:53:17 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:53:27.954226 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=28 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:55:24.344460 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ae6bac-8c93-40af-9404-63eb954df1ce service cinder] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:55:24.345924 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-63ae6bac-8c93-40af-9404-63eb954df1ce service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:55:24.435586 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ae6bac-8c93-40af-9404-63eb954df1ce service cinder] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:55:24.441104 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0edc4e-2c43-494d-ba29-dc7b0690a51d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '835cbc13756b41b598142df3e35f5492', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:55:24.441761 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0edc4e-2c43-494d-ba29-dc7b0690a51d service placement] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:55:24.442039 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0edc4e-2c43-494d-ba29-dc7b0690a51d service placement] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:55:24.442307 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0edc4e-2c43-494d-ba29-dc7b0690a51d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:55:24.451299 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0edc4e-2c43-494d-ba29-dc7b0690a51d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:55:24.452945 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0edc4e-2c43-494d-ba29-dc7b0690a51d service placement] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:55:24.463617 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1137/2265] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:55:24 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:55:34.458753 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=37 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:56:32.102772 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c553c4-aa3e-453a-bb13-c65a3fb721bd service cinder] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:56:32.155327 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c553c4-aa3e-453a-bb13-c65a3fb721bd service cinder] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:56:32.160831 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93a9b2c-5025-4a7e-88c6-4426793573a0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c69161cd520b4b29bf50f3bf2f1c033b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:56:32.161447 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a9b2c-5025-4a7e-88c6-4426793573a0 service nova] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:56:32.161764 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a9b2c-5025-4a7e-88c6-4426793573a0 service nova] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:56:32.162040 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a9b2c-5025-4a7e-88c6-4426793573a0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:56:32.171150 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93a9b2c-5025-4a7e-88c6-4426793573a0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:56:32.172679 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93a9b2c-5025-4a7e-88c6-4426793573a0 service nova] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:56:32.201092 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1129/2266] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:56:32 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:56:33.223344 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0edc4e-2c43-494d-ba29-dc7b0690a51d service placement] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:56:33.235340 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0edc4e-2c43-494d-ba29-dc7b0690a51d service placement] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:56:33.240723 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fda7c0-8480-4313-b12f-1f407e80c80c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c69161cd520b4b29bf50f3bf2f1c033b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:56:33.241262 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fda7c0-8480-4313-b12f-1f407e80c80c service nova] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:56:33.241575 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fda7c0-8480-4313-b12f-1f407e80c80c service nova] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:56:33.241848 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fda7c0-8480-4313-b12f-1f407e80c80c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:56:33.250296 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fda7c0-8480-4313-b12f-1f407e80c80c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:56:33.251905 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fda7c0-8480-4313-b12f-1f407e80c80c service nova] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:56:33.280591 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1138/2267] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:56:33 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:56:37.070392 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93a9b2c-5025-4a7e-88c6-4426793573a0 service nova] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:56:37.081399 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93a9b2c-5025-4a7e-88c6-4426793573a0 service nova] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:56:37.087297 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334922a2-fb7d-428d-bbbf-82d1c90125e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c69161cd520b4b29bf50f3bf2f1c033b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:56:37.087989 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334922a2-fb7d-428d-bbbf-82d1c90125e9 service nova] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:56:37.088271 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334922a2-fb7d-428d-bbbf-82d1c90125e9 service nova] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:56:37.088545 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334922a2-fb7d-428d-bbbf-82d1c90125e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:56:37.096651 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334922a2-fb7d-428d-bbbf-82d1c90125e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:56:37.098321 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334922a2-fb7d-428d-bbbf-82d1c90125e9 service nova] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:56:37.126352 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1130/2268] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:56:37 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:56:37.178200 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fda7c0-8480-4313-b12f-1f407e80c80c service nova] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:56:37.196866 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fda7c0-8480-4313-b12f-1f407e80c80c service nova] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:56:37.200904 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b960620-ef7a-4d30-a93f-2abdfce6c6dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:56:37.201538 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b960620-ef7a-4d30-a93f-2abdfce6c6dd service neutron] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:56:37.201820 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b960620-ef7a-4d30-a93f-2abdfce6c6dd service neutron] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:56:37.202090 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b960620-ef7a-4d30-a93f-2abdfce6c6dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:56:37.212351 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b960620-ef7a-4d30-a93f-2abdfce6c6dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:56:37.213774 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b960620-ef7a-4d30-a93f-2abdfce6c6dd service neutron] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:56:37.224971 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1139/2269] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:56:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:56:38.404484 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334922a2-fb7d-428d-bbbf-82d1c90125e9 service nova] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:56:38.413974 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334922a2-fb7d-428d-bbbf-82d1c90125e9 service nova] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:56:38.417597 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8f7af7-413b-4b13-8ac9-164384bf049d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:56:38.418211 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8f7af7-413b-4b13-8ac9-164384bf049d service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:56:38.418622 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8f7af7-413b-4b13-8ac9-164384bf049d service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:56:38.418845 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8f7af7-413b-4b13-8ac9-164384bf049d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:56:38.431974 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8f7af7-413b-4b13-8ac9-164384bf049d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:56:38.444556 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8f7af7-413b-4b13-8ac9-164384bf049d service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:56:38.456682 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1131/2270] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:56:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:56:38.523427 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b960620-ef7a-4d30-a93f-2abdfce6c6dd service neutron] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:56:38.549509 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b960620-ef7a-4d30-a93f-2abdfce6c6dd service neutron] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:56:38.562377 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f76916b-f629-4b3f-ae17-0379cb6716c4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c69161cd520b4b29bf50f3bf2f1c033b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:56:38.563026 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f76916b-f629-4b3f-ae17-0379cb6716c4 service nova] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:56:38.563377 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f76916b-f629-4b3f-ae17-0379cb6716c4 service nova] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:56:38.563706 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f76916b-f629-4b3f-ae17-0379cb6716c4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:56:38.565462 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-5f76916b-f629-4b3f-ae17-0379cb6716c4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 13:56:38.599057 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f76916b-f629-4b3f-ae17-0379cb6716c4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:56:38.601722 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f76916b-f629-4b3f-ae17-0379cb6716c4 service nova] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:56:38.619152 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1140/2271] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:56:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:56:48.451288 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=50 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:56:48.610363 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=34 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:57:26.955764 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8f7af7-413b-4b13-8ac9-164384bf049d service neutron] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:57:26.964494 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8f7af7-413b-4b13-8ac9-164384bf049d service neutron] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:57:26.968101 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdf709b-ffd0-49a5-a57f-0661950920b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:57:26.968707 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdf709b-ffd0-49a5-a57f-0661950920b3 service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:57:26.968987 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdf709b-ffd0-49a5-a57f-0661950920b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:57:26.969261 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdf709b-ffd0-49a5-a57f-0661950920b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:57:26.976394 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdf709b-ffd0-49a5-a57f-0661950920b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:57:26.977853 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdf709b-ffd0-49a5-a57f-0661950920b3 service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:57:26.985788 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1132/2272] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:57:26 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 31 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:57:36.354021 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f76916b-f629-4b3f-ae17-0379cb6716c4 service nova] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:57:36.372149 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f76916b-f629-4b3f-ae17-0379cb6716c4 service nova] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:57:36.375676 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5137055c-946d-4044-8279-e9f881d29bda service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:57:36.376281 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5137055c-946d-4044-8279-e9f881d29bda service cinder] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:57:36.376563 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5137055c-946d-4044-8279-e9f881d29bda service cinder] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:57:36.376841 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5137055c-946d-4044-8279-e9f881d29bda service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:57:36.385748 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5137055c-946d-4044-8279-e9f881d29bda service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:57:36.387143 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5137055c-946d-4044-8279-e9f881d29bda service cinder] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:57:36.398322 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1141/2273] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:57:36 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:57:36.982321 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=6 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:57:46.393191 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=10 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 13:58:17.159447 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdf709b-ffd0-49a5-a57f-0661950920b3 service neutron] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 13:58:17.172666 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdf709b-ffd0-49a5-a57f-0661950920b3 service neutron] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 13:58:17.178424 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8166f96e-26bb-46b5-a73f-b4c672087eaf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c69161cd520b4b29bf50f3bf2f1c033b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 13:58:17.179039 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8166f96e-26bb-46b5-a73f-b4c672087eaf service nova] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 13:58:17.179322 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8166f96e-26bb-46b5-a73f-b4c672087eaf service nova] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 13:58:17.179597 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8166f96e-26bb-46b5-a73f-b4c672087eaf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 13:58:17.188198 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8166f96e-26bb-46b5-a73f-b4c672087eaf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 13:58:17.189789 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8166f96e-26bb-46b5-a73f-b4c672087eaf service nova] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 13:58:17.217627 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1133/2274] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 13:58:17 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 13:58:27.215024 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=13 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:00:30.295642 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5137055c-946d-4044-8279-e9f881d29bda service cinder] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:00:30.374502 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5137055c-946d-4044-8279-e9f881d29bda service cinder] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:00:30.379657 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a43297-2222-4835-8e7c-ec552d9f2cfc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '835cbc13756b41b598142df3e35f5492', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:00:30.380276 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a43297-2222-4835-8e7c-ec552d9f2cfc service placement] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:00:30.380560 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a43297-2222-4835-8e7c-ec552d9f2cfc service placement] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:00:30.380834 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a43297-2222-4835-8e7c-ec552d9f2cfc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:00:30.382781 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-21a43297-2222-4835-8e7c-ec552d9f2cfc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:00:30.398388 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a43297-2222-4835-8e7c-ec552d9f2cfc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:00:30.401363 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a43297-2222-4835-8e7c-ec552d9f2cfc service placement] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:00:30.414399 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1142/2275] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:00:30 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:00:40.409969 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=32 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:01:31.668790 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8166f96e-26bb-46b5-a73f-b4c672087eaf service nova] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:01:31.725307 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8166f96e-26bb-46b5-a73f-b4c672087eaf service nova] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:01:31.728827 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7e671c-6826-4c1f-ba17-7cec9768206c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:01:31.729428 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7e671c-6826-4c1f-ba17-7cec9768206c service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:01:31.729736 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7e671c-6826-4c1f-ba17-7cec9768206c service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:01:31.730007 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7e671c-6826-4c1f-ba17-7cec9768206c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:01:31.738248 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7e671c-6826-4c1f-ba17-7cec9768206c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:01:31.739651 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7e671c-6826-4c1f-ba17-7cec9768206c service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:01:31.748472 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1134/2276] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:01:31 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:01:33.053361 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a43297-2222-4835-8e7c-ec552d9f2cfc service placement] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:01:33.073902 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a43297-2222-4835-8e7c-ec552d9f2cfc service placement] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:01:33.077710 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de25e71-bbd5-4578-b98a-15918a21728d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:01:33.078346 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de25e71-bbd5-4578-b98a-15918a21728d service cinder] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:01:33.078633 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de25e71-bbd5-4578-b98a-15918a21728d service cinder] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:01:33.078911 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de25e71-bbd5-4578-b98a-15918a21728d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:01:33.080827 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-5de25e71-bbd5-4578-b98a-15918a21728d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:01:33.168179 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de25e71-bbd5-4578-b98a-15918a21728d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:01:33.173856 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de25e71-bbd5-4578-b98a-15918a21728d service cinder] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:01:33.182114 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1143/2277] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:01:33 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:01:36.712170 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7e671c-6826-4c1f-ba17-7cec9768206c service cinder] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:01:36.723191 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7e671c-6826-4c1f-ba17-7cec9768206c service cinder] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:01:36.727300 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b100e9d-b4d8-4521-b480-1073f547fb51 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:01:36.727905 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b100e9d-b4d8-4521-b480-1073f547fb51 service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:01:36.728182 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b100e9d-b4d8-4521-b480-1073f547fb51 service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:01:36.728450 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b100e9d-b4d8-4521-b480-1073f547fb51 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:01:36.787686 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b100e9d-b4d8-4521-b480-1073f547fb51 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:01:36.789167 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b100e9d-b4d8-4521-b480-1073f547fb51 service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:01:36.799912 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1135/2278] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:01:36 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:01:38.997648 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de25e71-bbd5-4578-b98a-15918a21728d service cinder] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:01:38.999256 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-5de25e71-bbd5-4578-b98a-15918a21728d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:01:39.075913 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de25e71-bbd5-4578-b98a-15918a21728d service cinder] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:01:39.079578 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d51c2db-a14d-4eea-a4ca-6f7e3f9cc38c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:01:39.080195 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d51c2db-a14d-4eea-a4ca-6f7e3f9cc38c service neutron] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:01:39.080479 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d51c2db-a14d-4eea-a4ca-6f7e3f9cc38c service neutron] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:01:39.080753 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d51c2db-a14d-4eea-a4ca-6f7e3f9cc38c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:01:39.082672 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-9d51c2db-a14d-4eea-a4ca-6f7e3f9cc38c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:01:39.172253 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d51c2db-a14d-4eea-a4ca-6f7e3f9cc38c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:01:39.173632 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d51c2db-a14d-4eea-a4ca-6f7e3f9cc38c service neutron] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:01:39.187029 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1144/2279] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:01:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:01:46.794888 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=54 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:01:49.180719 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=92 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:02:37.321614 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b100e9d-b4d8-4521-b480-1073f547fb51 service cinder] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:02:37.323369 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-5b100e9d-b4d8-4521-b480-1073f547fb51 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:02:37.341518 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b100e9d-b4d8-4521-b480-1073f547fb51 service cinder] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:02:37.347429 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622121cf-c1ed-4575-b745-50e14af256ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c69161cd520b4b29bf50f3bf2f1c033b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:02:37.348600 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622121cf-c1ed-4575-b745-50e14af256ab service nova] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:02:37.349012 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622121cf-c1ed-4575-b745-50e14af256ab service nova] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:02:37.349233 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622121cf-c1ed-4575-b745-50e14af256ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:02:37.351427 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-622121cf-c1ed-4575-b745-50e14af256ab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:02:37.441956 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622121cf-c1ed-4575-b745-50e14af256ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:02:37.443645 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622121cf-c1ed-4575-b745-50e14af256ab service nova] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:02:37.463815 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1136/2280] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:02:37 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:02:42.367368 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d51c2db-a14d-4eea-a4ca-6f7e3f9cc38c service neutron] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:02:42.377111 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d51c2db-a14d-4eea-a4ca-6f7e3f9cc38c service neutron] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:02:42.382019 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde9a805-2b01-493d-8c38-d74b45ec1506 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:02:42.382673 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde9a805-2b01-493d-8c38-d74b45ec1506 service neutron] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:02:42.382957 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde9a805-2b01-493d-8c38-d74b45ec1506 service neutron] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:02:42.383233 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde9a805-2b01-493d-8c38-d74b45ec1506 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:02:42.385275 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-cde9a805-2b01-493d-8c38-d74b45ec1506 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:02:42.399927 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde9a805-2b01-493d-8c38-d74b45ec1506 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:02:42.401380 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde9a805-2b01-493d-8c38-d74b45ec1506 service neutron] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:02:42.413409 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1145/2281] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:02:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:02:43.444154 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622121cf-c1ed-4575-b745-50e14af256ab service nova] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:02:43.446368 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-622121cf-c1ed-4575-b745-50e14af256ab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:02:43.467268 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622121cf-c1ed-4575-b745-50e14af256ab service nova] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:02:43.476905 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a441809-65a4-4f22-8541-39fed1e85f53 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c69161cd520b4b29bf50f3bf2f1c033b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:02:43.478666 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a441809-65a4-4f22-8541-39fed1e85f53 service nova] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:02:43.479451 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a441809-65a4-4f22-8541-39fed1e85f53 service nova] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:02:43.480250 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a441809-65a4-4f22-8541-39fed1e85f53 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:02:43.500154 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a441809-65a4-4f22-8541-39fed1e85f53 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:02:43.504570 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a441809-65a4-4f22-8541-39fed1e85f53 service nova] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:02:43.521209 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1137/2282] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:02:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:02:52.408040 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=6 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:02:53.515344 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=47 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:03:18.473459 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde9a805-2b01-493d-8c38-d74b45ec1506 service neutron] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:03:18.486285 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde9a805-2b01-493d-8c38-d74b45ec1506 service neutron] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:03:18.493379 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03948d55-d402-4516-8cd4-266e76e878bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c69161cd520b4b29bf50f3bf2f1c033b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:03:18.494060 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03948d55-d402-4516-8cd4-266e76e878bc service nova] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:03:18.494371 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03948d55-d402-4516-8cd4-266e76e878bc service nova] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:03:18.494648 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03948d55-d402-4516-8cd4-266e76e878bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:03:18.496608 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-03948d55-d402-4516-8cd4-266e76e878bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:03:18.569991 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03948d55-d402-4516-8cd4-266e76e878bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:03:18.571671 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03948d55-d402-4516-8cd4-266e76e878bc service nova] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:03:18.580700 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1146/2283] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:03:18 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:03:28.576486 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=33 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:03:35.627992 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a441809-65a4-4f22-8541-39fed1e85f53 service nova] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:03:35.640679 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a441809-65a4-4f22-8541-39fed1e85f53 service nova] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:03:35.645290 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e34b05-f8c1-4507-92d1-b8cd2b8cf5e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:03:35.645963 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e34b05-f8c1-4507-92d1-b8cd2b8cf5e2 service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:03:35.646240 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e34b05-f8c1-4507-92d1-b8cd2b8cf5e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:03:35.646511 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e34b05-f8c1-4507-92d1-b8cd2b8cf5e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:03:35.649513 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-66e34b05-f8c1-4507-92d1-b8cd2b8cf5e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:03:35.663364 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e34b05-f8c1-4507-92d1-b8cd2b8cf5e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:03:35.666428 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e34b05-f8c1-4507-92d1-b8cd2b8cf5e2 service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:03:35.677645 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1138/2284] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:03:35 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:03:45.673079 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=6 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:05:33.294116 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03948d55-d402-4516-8cd4-266e76e878bc service nova] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:05:33.298235 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-03948d55-d402-4516-8cd4-266e76e878bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:05:33.423447 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03948d55-d402-4516-8cd4-266e76e878bc service nova] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:05:33.429154 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91524df2-2ff8-4f37-bd35-1358d0d5ca0c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '835cbc13756b41b598142df3e35f5492', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:05:33.429973 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91524df2-2ff8-4f37-bd35-1358d0d5ca0c service placement] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:05:33.430436 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91524df2-2ff8-4f37-bd35-1358d0d5ca0c service placement] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:05:33.430833 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91524df2-2ff8-4f37-bd35-1358d0d5ca0c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:05:33.441001 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91524df2-2ff8-4f37-bd35-1358d0d5ca0c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:05:33.442662 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91524df2-2ff8-4f37-bd35-1358d0d5ca0c service placement] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:05:33.453575 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1147/2285] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:05:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 14:05:43.448763 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=26 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:06:31.619754 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e34b05-f8c1-4507-92d1-b8cd2b8cf5e2 service neutron] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:06:31.645250 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e34b05-f8c1-4507-92d1-b8cd2b8cf5e2 service neutron] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:06:31.654582 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc067b3-9f1c-466f-a891-5c9b0d081ba6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:06:31.656311 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc067b3-9f1c-466f-a891-5c9b0d081ba6 service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:06:31.656699 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc067b3-9f1c-466f-a891-5c9b0d081ba6 service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:06:31.657150 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc067b3-9f1c-466f-a891-5c9b0d081ba6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:06:31.659678 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-ffc067b3-9f1c-466f-a891-5c9b0d081ba6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:06:31.812553 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc067b3-9f1c-466f-a891-5c9b0d081ba6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:06:31.815858 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc067b3-9f1c-466f-a891-5c9b0d081ba6 service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:06:31.828688 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1139/2286] 10.222.0.58 () {60 vars in 1396 bytes} [Tue Jul 26 14:06:31 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3254 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:06:32.831822 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91524df2-2ff8-4f37-bd35-1358d0d5ca0c service placement] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:06:32.920334 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91524df2-2ff8-4f37-bd35-1358d0d5ca0c service placement] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:06:32.927625 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887473b8-4b2c-482a-85ff-f33545650f72 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c69161cd520b4b29bf50f3bf2f1c033b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:06:32.928394 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887473b8-4b2c-482a-85ff-f33545650f72 service nova] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:06:32.928779 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887473b8-4b2c-482a-85ff-f33545650f72 service nova] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:06:32.929180 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887473b8-4b2c-482a-85ff-f33545650f72 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:06:32.940541 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887473b8-4b2c-482a-85ff-f33545650f72 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:06:32.942379 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887473b8-4b2c-482a-85ff-f33545650f72 service nova] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:06:32.960871 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1148/2287] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:06:32 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:06:38.838041 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc067b3-9f1c-466f-a891-5c9b0d081ba6 service neutron] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:06:38.842858 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-ffc067b3-9f1c-466f-a891-5c9b0d081ba6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:06:38.901455 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc067b3-9f1c-466f-a891-5c9b0d081ba6 service neutron] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:06:38.907691 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cb037f-a689-4a44-a980-4f032a036c3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:06:38.908849 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cb037f-a689-4a44-a980-4f032a036c3b service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:06:38.909325 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cb037f-a689-4a44-a980-4f032a036c3b service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:06:38.909813 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cb037f-a689-4a44-a980-4f032a036c3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:06:38.912925 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-e9cb037f-a689-4a44-a980-4f032a036c3b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:06:38.971899 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cb037f-a689-4a44-a980-4f032a036c3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:06:38.974058 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cb037f-a689-4a44-a980-4f032a036c3b service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:06:39.006075 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1140/2288] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:06:38 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 14:06:39.200039 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887473b8-4b2c-482a-85ff-f33545650f72 service nova] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:06:39.237509 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887473b8-4b2c-482a-85ff-f33545650f72 service nova] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:06:39.242743 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86546fd-4679-49a3-8a8b-3e3ecda99629 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:06:39.243611 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86546fd-4679-49a3-8a8b-3e3ecda99629 service neutron] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:06:39.243980 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86546fd-4679-49a3-8a8b-3e3ecda99629 service neutron] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:06:39.244400 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86546fd-4679-49a3-8a8b-3e3ecda99629 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:06:39.261511 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86546fd-4679-49a3-8a8b-3e3ecda99629 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:06:39.262985 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86546fd-4679-49a3-8a8b-3e3ecda99629 service neutron] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:06:39.288857 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1149/2289] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:06:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:06:49.001234 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=62 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:06:49.269961 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=34 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:07:37.898599 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cb037f-a689-4a44-a980-4f032a036c3b service cinder] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:07:37.917151 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cb037f-a689-4a44-a980-4f032a036c3b service cinder] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:07:37.924666 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ef5870-fba5-4c03-946b-db9ba8c89be1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c69161cd520b4b29bf50f3bf2f1c033b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:07:37.925374 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ef5870-fba5-4c03-946b-db9ba8c89be1 service nova] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:07:37.925733 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ef5870-fba5-4c03-946b-db9ba8c89be1 service nova] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:07:37.926011 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ef5870-fba5-4c03-946b-db9ba8c89be1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:07:37.940943 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ef5870-fba5-4c03-946b-db9ba8c89be1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:07:37.943607 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ef5870-fba5-4c03-946b-db9ba8c89be1 service nova] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:07:37.959625 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1141/2290] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:07:37 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 14:07:42.707841 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86546fd-4679-49a3-8a8b-3e3ecda99629 service neutron] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:07:42.723646 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86546fd-4679-49a3-8a8b-3e3ecda99629 service neutron] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:07:42.727451 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682d39b2-daa8-4c5d-a251-7d6bb7531229 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:07:42.728396 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682d39b2-daa8-4c5d-a251-7d6bb7531229 service neutron] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:07:42.728868 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682d39b2-daa8-4c5d-a251-7d6bb7531229 service neutron] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:07:42.729336 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682d39b2-daa8-4c5d-a251-7d6bb7531229 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:07:42.749381 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682d39b2-daa8-4c5d-a251-7d6bb7531229 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:07:42.751094 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682d39b2-daa8-4c5d-a251-7d6bb7531229 service neutron] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:07:42.773928 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1150/2291] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:07:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 14:07:47.952277 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=6 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:07:52.757837 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=6 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:08:39.381552 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ef5870-fba5-4c03-946b-db9ba8c89be1 service nova] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:08:39.415567 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ef5870-fba5-4c03-946b-db9ba8c89be1 service nova] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:08:39.421277 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58f792c-6a94-41f9-9c9a-e088d7dc0f0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:08:39.422234 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58f792c-6a94-41f9-9c9a-e088d7dc0f0a service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:08:39.422519 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58f792c-6a94-41f9-9c9a-e088d7dc0f0a service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:08:39.422948 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58f792c-6a94-41f9-9c9a-e088d7dc0f0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:08:39.438803 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58f792c-6a94-41f9-9c9a-e088d7dc0f0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:08:39.441351 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58f792c-6a94-41f9-9c9a-e088d7dc0f0a service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:08:39.481997 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1142/2292] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:08:39 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:08:40.007740 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682d39b2-daa8-4c5d-a251-7d6bb7531229 service neutron] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:08:40.022973 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682d39b2-daa8-4c5d-a251-7d6bb7531229 service neutron] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:08:40.029323 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7449b1c0-debf-4476-a5a4-f216cdb208a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:08:40.030022 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7449b1c0-debf-4476-a5a4-f216cdb208a6 service neutron] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:08:40.030312 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7449b1c0-debf-4476-a5a4-f216cdb208a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:08:40.030599 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7449b1c0-debf-4476-a5a4-f216cdb208a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:08:40.059648 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7449b1c0-debf-4476-a5a4-f216cdb208a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:08:40.063866 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7449b1c0-debf-4476-a5a4-f216cdb208a6 service neutron] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:08:40.086070 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1151/2293] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:08:40 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:08:44.408175 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bcab05-566b-419b-b7f7-c6e451628ef0 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:08:44.409127 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bcab05-566b-419b-b7f7-c6e451628ef0 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:08:44.409770 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bcab05-566b-419b-b7f7-c6e451628ef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:08:44.482874 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-d5bcab05-566b-419b-b7f7-c6e451628ef0 None None] MFA Rules not processed for user `56db6603f359477885b5151dec71c096`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:08:44.484126 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-d5bcab05-566b-419b-b7f7-c6e451628ef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:08:44.506503 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1143/2294] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:08:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:08:44.896487 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7449b1c0-debf-4476-a5a4-f216cdb208a6 service neutron] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:08:44.905704 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7449b1c0-debf-4476-a5a4-f216cdb208a6 service neutron] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:08:44.909521 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f2f996-3529-4b40-8d93-c7279a195098 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:08:44.910139 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f2f996-3529-4b40-8d93-c7279a195098 service neutron] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:08:44.910421 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f2f996-3529-4b40-8d93-c7279a195098 service neutron] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:08:44.910698 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f2f996-3529-4b40-8d93-c7279a195098 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:08:44.912516 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-94f2f996-3529-4b40-8d93-c7279a195098 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:08:44.922851 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f2f996-3529-4b40-8d93-c7279a195098 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:08:44.924271 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f2f996-3529-4b40-8d93-c7279a195098 service neutron] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:08:44.933055 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1152/2295] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:08:44 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:08:45.425244 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bcab05-566b-419b-b7f7-c6e451628ef0 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:08:45.438449 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bcab05-566b-419b-b7f7-c6e451628ef0 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:08:45.443837 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3452334c-e4cb-42f7-b57e-1bf266e92434 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '835cbc13756b41b598142df3e35f5492', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:08:45.444482 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3452334c-e4cb-42f7-b57e-1bf266e92434 service placement] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:08:45.444764 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3452334c-e4cb-42f7-b57e-1bf266e92434 service placement] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:08:45.445047 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3452334c-e4cb-42f7-b57e-1bf266e92434 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:08:45.446974 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-3452334c-e4cb-42f7-b57e-1bf266e92434 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:08:45.462656 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3452334c-e4cb-42f7-b57e-1bf266e92434 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:08:45.464276 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3452334c-e4cb-42f7-b57e-1bf266e92434 service placement] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:08:45.480436 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1144/2296] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:08:45 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:08:45.804583 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f2f996-3529-4b40-8d93-c7279a195098 service neutron] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:08:45.835477 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f2f996-3529-4b40-8d93-c7279a195098 service neutron] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:08:45.842514 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213734aa-40a4-4ed7-b184-7add51404892 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c69161cd520b4b29bf50f3bf2f1c033b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:08:45.844580 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213734aa-40a4-4ed7-b184-7add51404892 service nova] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:08:45.845357 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213734aa-40a4-4ed7-b184-7add51404892 service nova] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:08:45.846300 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213734aa-40a4-4ed7-b184-7add51404892 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:08:45.849755 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-213734aa-40a4-4ed7-b184-7add51404892 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:08:45.918680 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213734aa-40a4-4ed7-b184-7add51404892 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:08:45.920289 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213734aa-40a4-4ed7-b184-7add51404892 service nova] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:08:45.960705 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1153/2297] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:08:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:08:48.403081 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3452334c-e4cb-42f7-b57e-1bf266e92434 service placement] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:08:48.417445 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3452334c-e4cb-42f7-b57e-1bf266e92434 service placement] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:08:48.424914 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40896e1-7436-475b-89ec-50399a5fb623 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c69161cd520b4b29bf50f3bf2f1c033b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:08:48.425661 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40896e1-7436-475b-89ec-50399a5fb623 service nova] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:08:48.426248 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40896e1-7436-475b-89ec-50399a5fb623 service nova] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:08:48.426697 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40896e1-7436-475b-89ec-50399a5fb623 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:08:48.428747 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-a40896e1-7436-475b-89ec-50399a5fb623 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:08:48.447553 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40896e1-7436-475b-89ec-50399a5fb623 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:08:48.449739 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40896e1-7436-475b-89ec-50399a5fb623 service nova] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:08:48.469957 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1145/2298] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:08:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:08:49.661995 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213734aa-40a4-4ed7-b184-7add51404892 service nova] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:08:49.663671 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-213734aa-40a4-4ed7-b184-7add51404892 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:08:49.675726 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213734aa-40a4-4ed7-b184-7add51404892 service nova] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:08:49.679681 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d01934-d7d4-42ef-9e2b-8c6ebc0f91f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:08:49.680310 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d01934-d7d4-42ef-9e2b-8c6ebc0f91f4 service neutron] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:08:49.680598 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d01934-d7d4-42ef-9e2b-8c6ebc0f91f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:08:49.680873 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d01934-d7d4-42ef-9e2b-8c6ebc0f91f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:08:49.682972 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-b5d01934-d7d4-42ef-9e2b-8c6ebc0f91f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:08:49.775941 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d01934-d7d4-42ef-9e2b-8c6ebc0f91f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:08:49.777389 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d01934-d7d4-42ef-9e2b-8c6ebc0f91f4 service neutron] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:08:49.799421 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1154/2299] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:08:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:08:53.690748 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40896e1-7436-475b-89ec-50399a5fb623 service nova] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:08:53.713531 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40896e1-7436-475b-89ec-50399a5fb623 service nova] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:08:53.720540 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481d5592-8b98-4630-82de-ac73852f0726 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '835cbc13756b41b598142df3e35f5492', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:08:53.721502 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481d5592-8b98-4630-82de-ac73852f0726 service placement] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:08:53.722117 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481d5592-8b98-4630-82de-ac73852f0726 service placement] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:08:53.722552 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481d5592-8b98-4630-82de-ac73852f0726 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:08:53.728559 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-481d5592-8b98-4630-82de-ac73852f0726 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:08:53.754794 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481d5592-8b98-4630-82de-ac73852f0726 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:08:53.756945 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481d5592-8b98-4630-82de-ac73852f0726 service placement] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:08:53.773460 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1146/2300] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:08:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:08:53.787316 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d01934-d7d4-42ef-9e2b-8c6ebc0f91f4 service neutron] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:08:53.790400 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-b5d01934-d7d4-42ef-9e2b-8c6ebc0f91f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:08:53.819141 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d01934-d7d4-42ef-9e2b-8c6ebc0f91f4 service neutron] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:08:53.830747 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad08dcd7-a3a8-44ae-bda1-3de391b63e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:08:53.832425 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad08dcd7-a3a8-44ae-bda1-3de391b63e88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:08:53.832938 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad08dcd7-a3a8-44ae-bda1-3de391b63e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:08:53.833450 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad08dcd7-a3a8-44ae-bda1-3de391b63e88 admin admin] PATH_INFO: `/v3/users/0accc3551cd348e2a946bde38e4db40f` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:08:53.835779 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad08dcd7-a3a8-44ae-bda1-3de391b63e88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0accc3551cd348e2a946bde38e4db40f)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:08:53.839170 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad08dcd7-a3a8-44ae-bda1-3de391b63e88 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:08:53.915526 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-ad08dcd7-a3a8-44ae-bda1-3de391b63e88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0accc3551cd348e2a946bde38e4db40f'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:08:53.925841 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-ad08dcd7-a3a8-44ae-bda1-3de391b63e88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0accc3551cd348e2a946bde38e4db40f'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:08:53.935179 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-ad08dcd7-a3a8-44ae-bda1-3de391b63e88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0accc3551cd348e2a946bde38e4db40f'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:08:53.936085 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-ad08dcd7-a3a8-44ae-bda1-3de391b63e88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0accc3551cd348e2a946bde38e4db40f'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:08:53.936751 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-ad08dcd7-a3a8-44ae-bda1-3de391b63e88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0accc3551cd348e2a946bde38e4db40f'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:08:53.950977 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:08:53.959342 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1155/2301] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 14:08:53 2022] DELETE /identity/v3/users/0accc3551cd348e2a946bde38e4db40f => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:08:54.158142 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481d5592-8b98-4630-82de-ac73852f0726 service placement] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:08:54.175804 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481d5592-8b98-4630-82de-ac73852f0726 service placement] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:08:54.259198 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fabd2a1-3a6f-492b-8a33-2a5758bcbfe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:08:54.259855 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fabd2a1-3a6f-492b-8a33-2a5758bcbfe0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:08:54.260143 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fabd2a1-3a6f-492b-8a33-2a5758bcbfe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:08:54.260429 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fabd2a1-3a6f-492b-8a33-2a5758bcbfe0 admin admin] PATH_INFO: `/v3/projects/78ae845c827f4448ad340221d5816fd1` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:08:54.261625 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fabd2a1-3a6f-492b-8a33-2a5758bcbfe0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78ae845c827f4448ad340221d5816fd1)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:08:54.263538 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fabd2a1-3a6f-492b-8a33-2a5758bcbfe0 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:08:54.320266 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-9fabd2a1-3a6f-492b-8a33-2a5758bcbfe0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78ae845c827f4448ad340221d5816fd1', 'request_id': 'req-9fabd2a1-3a6f-492b-8a33-2a5758bcbfe0'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:08:54.326859 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-9fabd2a1-3a6f-492b-8a33-2a5758bcbfe0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78ae845c827f4448ad340221d5816fd1', 'request_id': 'req-9fabd2a1-3a6f-492b-8a33-2a5758bcbfe0'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:08:54.328156 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:08:54.333868 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-9fabd2a1-3a6f-492b-8a33-2a5758bcbfe0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78ae845c827f4448ad340221d5816fd1', 'request_id': 'req-9fabd2a1-3a6f-492b-8a33-2a5758bcbfe0'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:08:54.348265 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-9fabd2a1-3a6f-492b-8a33-2a5758bcbfe0 admin admin] The token cache is being invalidate because project 78ae845c827f4448ad340221d5816fd1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:08:54.348554 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-9fabd2a1-3a6f-492b-8a33-2a5758bcbfe0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:08:54.349220 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-9fabd2a1-3a6f-492b-8a33-2a5758bcbfe0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:08:54.350818 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1147/2302] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 14:08:54 2022] DELETE /identity/v3/projects/78ae845c827f4448ad340221d5816fd1 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:08:54.356652 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae92bd69-e96b-447a-8d75-d37ea691b983 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:08:54.357000 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae92bd69-e96b-447a-8d75-d37ea691b983 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:08:54.357309 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae92bd69-e96b-447a-8d75-d37ea691b983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:08:54.435809 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-ae92bd69-e96b-447a-8d75-d37ea691b983 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:08:54.437864 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-ae92bd69-e96b-447a-8d75-d37ea691b983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:08:54.537420 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1156/2303] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:08:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:08:54.552242 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e0f879-d711-4946-9dd6-40512d7568d7 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:08:54.552700 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e0f879-d711-4946-9dd6-40512d7568d7 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:08:54.552988 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e0f879-d711-4946-9dd6-40512d7568d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:08:54.575301 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-e2e0f879-d711-4946-9dd6-40512d7568d7 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:08:54.576717 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-e2e0f879-d711-4946-9dd6-40512d7568d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:08:54.587403 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1148/2304] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:08:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:08:54.596377 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae92bd69-e96b-447a-8d75-d37ea691b983 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:08:54.610527 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae92bd69-e96b-447a-8d75-d37ea691b983 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:08:54.615548 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50d6083-0cb5-4d0e-8830-b424523bea43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:08:54.616195 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50d6083-0cb5-4d0e-8830-b424523bea43 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:08:54.616479 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50d6083-0cb5-4d0e-8830-b424523bea43 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:08:54.616757 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50d6083-0cb5-4d0e-8830-b424523bea43 admin admin] PATH_INFO: `/v3/domains` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:08:54.617545 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50d6083-0cb5-4d0e-8830-b424523bea43 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:08:54.617837 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50d6083-0cb5-4d0e-8830-b424523bea43 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:08:54.619456 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50d6083-0cb5-4d0e-8830-b424523bea43 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:08:54.634264 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1157/2305] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Jul 26 14:08:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:08:54.641713 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e0f879-d711-4946-9dd6-40512d7568d7 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:08:54.654784 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e0f879-d711-4946-9dd6-40512d7568d7 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:08:54.661653 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38fe076-d9c2-4a3e-bcd2-f626713cd66a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:08:54.662580 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38fe076-d9c2-4a3e-bcd2-f626713cd66a admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:08:54.662863 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38fe076-d9c2-4a3e-bcd2-f626713cd66a admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:08:54.663136 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38fe076-d9c2-4a3e-bcd2-f626713cd66a admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:08:54.663816 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38fe076-d9c2-4a3e-bcd2-f626713cd66a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:08:54.666083 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38fe076-d9c2-4a3e-bcd2-f626713cd66a admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:08:54.716729 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1149/2306] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 14:08:54 2022] POST /identity/v3/projects => generated 355 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:08:54.721718 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50d6083-0cb5-4d0e-8830-b424523bea43 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:08:54.735872 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50d6083-0cb5-4d0e-8830-b424523bea43 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:08:54.812265 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ca10b1-cc1f-4285-b653-7bb8784b5b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:08:54.812981 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ca10b1-cc1f-4285-b653-7bb8784b5b6b admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:08:54.813229 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ca10b1-cc1f-4285-b653-7bb8784b5b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:08:54.813548 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ca10b1-cc1f-4285-b653-7bb8784b5b6b admin admin] PATH_INFO: `/v3/users` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:08:54.814196 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ca10b1-cc1f-4285-b653-7bb8784b5b6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:08:54.816202 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ca10b1-cc1f-4285-b653-7bb8784b5b6b admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:08:54.843855 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1158/2307] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 14:08:54 2022] POST /identity/v3/users => generated 278 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:08:54.848805 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38fe076-d9c2-4a3e-bcd2-f626713cd66a admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:08:54.859891 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38fe076-d9c2-4a3e-bcd2-f626713cd66a admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:08:54.865965 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12442b69-90fd-4498-90f7-ae84d655cb3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:08:54.866816 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12442b69-90fd-4498-90f7-ae84d655cb3e admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:08:54.867220 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12442b69-90fd-4498-90f7-ae84d655cb3e admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:08:54.867516 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12442b69-90fd-4498-90f7-ae84d655cb3e admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:08:54.869212 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12442b69-90fd-4498-90f7-ae84d655cb3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:08:54.870897 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12442b69-90fd-4498-90f7-ae84d655cb3e admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:08:54.882988 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1150/2308] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:08:54 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:08:54.887228 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ca10b1-cc1f-4285-b653-7bb8784b5b6b admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:08:54.907918 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ca10b1-cc1f-4285-b653-7bb8784b5b6b admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:08:54.914881 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9128a848-2ed7-44a3-9e3d-a42f7c896828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:08:54.915742 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9128a848-2ed7-44a3-9e3d-a42f7c896828 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:08:54.916226 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9128a848-2ed7-44a3-9e3d-a42f7c896828 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:08:54.916639 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9128a848-2ed7-44a3-9e3d-a42f7c896828 admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:08:54.917450 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9128a848-2ed7-44a3-9e3d-a42f7c896828 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:08:54.919197 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9128a848-2ed7-44a3-9e3d-a42f7c896828 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:08:54.937870 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1159/2309] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:08:54 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:08:54.939541 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12442b69-90fd-4498-90f7-ae84d655cb3e admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:08:54.954160 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12442b69-90fd-4498-90f7-ae84d655cb3e admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:08:54.961287 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed17433e-1f67-4e0f-94c0-022c35ccf16f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:08:54.962575 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed17433e-1f67-4e0f-94c0-022c35ccf16f admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:08:54.962944 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed17433e-1f67-4e0f-94c0-022c35ccf16f admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:08:54.963233 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed17433e-1f67-4e0f-94c0-022c35ccf16f admin admin] PATH_INFO: `/v3/projects/0307fce6884f440cba34f8db33fd956f/users/df107ff94ddc4b7c889dfcdfeb746b47/roles/759f389b221049c4b01faf3b984cd8d7` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:08:54.982060 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed17433e-1f67-4e0f-94c0-022c35ccf16f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0307fce6884f440cba34f8db33fd956f, user_id=df107ff94ddc4b7c889dfcdfeb746b47, role_id=759f389b221049c4b01faf3b984cd8d7)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:08:54.984965 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed17433e-1f67-4e0f-94c0-022c35ccf16f admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:08:55.011861 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1151/2310] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 14:08:54 2022] PUT /identity/v3/projects/0307fce6884f440cba34f8db33fd956f/users/df107ff94ddc4b7c889dfcdfeb746b47/roles/759f389b221049c4b01faf3b984cd8d7 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:08:55.024731 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a496ecc-2af0-4b89-99ae-9d79bc7e0348 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:08:55.024959 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a496ecc-2af0-4b89-99ae-9d79bc7e0348 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:08:55.025144 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a496ecc-2af0-4b89-99ae-9d79bc7e0348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:08:55.057223 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-5a496ecc-2af0-4b89-99ae-9d79bc7e0348 None None] MFA Rules not processed for user `df107ff94ddc4b7c889dfcdfeb746b47`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:08:55.058499 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-5a496ecc-2af0-4b89-99ae-9d79bc7e0348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:08:55.070138 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:08:55.071567 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed17433e-1f67-4e0f-94c0-022c35ccf16f admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:08:55.073866 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-ed17433e-1f67-4e0f-94c0-022c35ccf16f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:08:55.147163 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1160/2311] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:08:55 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:08:55.151702 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a496ecc-2af0-4b89-99ae-9d79bc7e0348 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:08:55.178290 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed17433e-1f67-4e0f-94c0-022c35ccf16f admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:08:55.187384 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f769fd4-cb23-44c7-9500-54e2d5e18666 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c69161cd520b4b29bf50f3bf2f1c033b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:08:55.188415 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f769fd4-cb23-44c7-9500-54e2d5e18666 service nova] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:08:55.188930 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f769fd4-cb23-44c7-9500-54e2d5e18666 service nova] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:08:55.189410 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f769fd4-cb23-44c7-9500-54e2d5e18666 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:08:55.192447 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-1f769fd4-cb23-44c7-9500-54e2d5e18666 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:08:55.213233 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f769fd4-cb23-44c7-9500-54e2d5e18666 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:08:55.216243 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f769fd4-cb23-44c7-9500-54e2d5e18666 service nova] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:08:55.227753 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1152/2312] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:08:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 14:08:55.233554 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a496ecc-2af0-4b89-99ae-9d79bc7e0348 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:08:55.238625 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d14265-5b0f-4790-a774-39dc807915f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:08:55.239701 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d14265-5b0f-4790-a774-39dc807915f5 admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:08:55.239701 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d14265-5b0f-4790-a774-39dc807915f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:08:55.240014 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d14265-5b0f-4790-a774-39dc807915f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:08:55.240772 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d14265-5b0f-4790-a774-39dc807915f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:08:55.243261 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d14265-5b0f-4790-a774-39dc807915f5 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:08:55.278420 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1161/2313] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 14:08:55 2022] POST /identity/v3/projects => generated 355 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 14:08:55.283294 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f769fd4-cb23-44c7-9500-54e2d5e18666 service nova] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:08:55.297264 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f769fd4-cb23-44c7-9500-54e2d5e18666 service nova] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:08:55.362615 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:08:55.375500 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32416147-733b-40ce-9438-bc53463170dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:08:55.376328 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32416147-733b-40ce-9438-bc53463170dd admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:08:55.376669 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32416147-733b-40ce-9438-bc53463170dd admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:08:55.377251 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32416147-733b-40ce-9438-bc53463170dd admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:08:55.377976 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32416147-733b-40ce-9438-bc53463170dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:08:55.379926 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32416147-733b-40ce-9438-bc53463170dd admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:08:55.406077 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1153/2314] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 14:08:55 2022] POST /identity/v3/users => generated 284 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:08:55.410307 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d14265-5b0f-4790-a774-39dc807915f5 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:08:55.422890 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d14265-5b0f-4790-a774-39dc807915f5 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:08:55.427866 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67a1106-cf55-4100-987b-3f07da5a9db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:08:55.428513 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67a1106-cf55-4100-987b-3f07da5a9db4 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:08:55.428863 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67a1106-cf55-4100-987b-3f07da5a9db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:08:55.429095 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67a1106-cf55-4100-987b-3f07da5a9db4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:08:55.429839 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67a1106-cf55-4100-987b-3f07da5a9db4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:08:55.431438 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67a1106-cf55-4100-987b-3f07da5a9db4 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:08:55.446747 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1162/2315] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:08:55 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:08:55.450878 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32416147-733b-40ce-9438-bc53463170dd admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:08:55.462179 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32416147-733b-40ce-9438-bc53463170dd admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:08:55.467038 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5710d6-c12a-4463-9bea-e7c64a177353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:08:55.467784 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5710d6-c12a-4463-9bea-e7c64a177353 admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:08:55.468157 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5710d6-c12a-4463-9bea-e7c64a177353 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:08:55.468434 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5710d6-c12a-4463-9bea-e7c64a177353 admin admin] PATH_INFO: `/v3/projects/2d158719c484419e857674c8877b83ca/users/f0cf3bb2c3f946938f1dbfd42feae40e/roles/6e0201a9a6484f7d8b112b1eafff70bd` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:08:55.491626 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5710d6-c12a-4463-9bea-e7c64a177353 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d158719c484419e857674c8877b83ca, user_id=f0cf3bb2c3f946938f1dbfd42feae40e, role_id=6e0201a9a6484f7d8b112b1eafff70bd)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:08:55.494417 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5710d6-c12a-4463-9bea-e7c64a177353 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:08:55.511520 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1154/2316] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 14:08:55 2022] PUT /identity/v3/projects/2d158719c484419e857674c8877b83ca/users/f0cf3bb2c3f946938f1dbfd42feae40e/roles/6e0201a9a6484f7d8b112b1eafff70bd => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:08:55.516239 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67a1106-cf55-4100-987b-3f07da5a9db4 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:08:55.529093 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67a1106-cf55-4100-987b-3f07da5a9db4 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:08:55.598088 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1223c7c6-27eb-45fe-af03-a0e8d66b891b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:08:55.598729 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1223c7c6-27eb-45fe-af03-a0e8d66b891b admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:08:55.599011 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1223c7c6-27eb-45fe-af03-a0e8d66b891b admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:08:55.599287 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1223c7c6-27eb-45fe-af03-a0e8d66b891b admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:08:55.599991 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1223c7c6-27eb-45fe-af03-a0e8d66b891b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:08:55.601667 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1223c7c6-27eb-45fe-af03-a0e8d66b891b admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:08:55.614486 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1163/2317] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:08:55 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 14:08:55.618855 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5710d6-c12a-4463-9bea-e7c64a177353 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:08:55.632649 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5710d6-c12a-4463-9bea-e7c64a177353 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:08:55.638093 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5765ea72-7c0b-4bf4-8f94-7e897a9e5b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:08:55.638783 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5765ea72-7c0b-4bf4-8f94-7e897a9e5b12 admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:08:55.639064 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5765ea72-7c0b-4bf4-8f94-7e897a9e5b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:08:55.639338 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5765ea72-7c0b-4bf4-8f94-7e897a9e5b12 admin admin] PATH_INFO: `/v3/projects/2d158719c484419e857674c8877b83ca/users/f0cf3bb2c3f946938f1dbfd42feae40e/roles/6e0201a9a6484f7d8b112b1eafff70bd` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:08:55.641871 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5765ea72-7c0b-4bf4-8f94-7e897a9e5b12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d158719c484419e857674c8877b83ca, user_id=f0cf3bb2c3f946938f1dbfd42feae40e, role_id=6e0201a9a6484f7d8b112b1eafff70bd)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:08:55.646144 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5765ea72-7c0b-4bf4-8f94-7e897a9e5b12 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:08:55.657760 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1155/2318] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 14:08:55 2022] PUT /identity/v3/projects/2d158719c484419e857674c8877b83ca/users/f0cf3bb2c3f946938f1dbfd42feae40e/roles/6e0201a9a6484f7d8b112b1eafff70bd => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:08:55.669890 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da61da2-f8a8-4a87-8c3d-ad6484e059cd None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:08:55.670221 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da61da2-f8a8-4a87-8c3d-ad6484e059cd None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:08:55.670462 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da61da2-f8a8-4a87-8c3d-ad6484e059cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:08:55.704227 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-8da61da2-f8a8-4a87-8c3d-ad6484e059cd None None] MFA Rules not processed for user `f0cf3bb2c3f946938f1dbfd42feae40e`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:08:55.705939 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-8da61da2-f8a8-4a87-8c3d-ad6484e059cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:08:55.723618 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:08:55.805059 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1164/2319] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:08:55 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:09:05.653314 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=19,keystone:INSERT=5 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:09:05.800041 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=24 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:10:40.241628 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5765ea72-7c0b-4bf4-8f94-7e897a9e5b12 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:10:40.243882 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-5765ea72-7c0b-4bf4-8f94-7e897a9e5b12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:10:40.339929 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da61da2-f8a8-4a87-8c3d-ad6484e059cd None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:10:40.342630 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-8da61da2-f8a8-4a87-8c3d-ad6484e059cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:10:40.385291 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5765ea72-7c0b-4bf4-8f94-7e897a9e5b12 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:10:40.393850 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ef4ac5-16de-4586-a063-3b151e0fb8b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:10:40.394910 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ef4ac5-16de-4586-a063-3b151e0fb8b0 service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:10:40.395403 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ef4ac5-16de-4586-a063-3b151e0fb8b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:10:40.396123 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ef4ac5-16de-4586-a063-3b151e0fb8b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:10:40.417878 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ef4ac5-16de-4586-a063-3b151e0fb8b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:10:40.421757 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ef4ac5-16de-4586-a063-3b151e0fb8b0 service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:10:40.446531 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1156/2320] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:10:40 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 14:10:40.483182 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da61da2-f8a8-4a87-8c3d-ad6484e059cd None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:10:40.492921 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11ffd92-2fbb-46fa-b40b-274937a24084 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '835cbc13756b41b598142df3e35f5492', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:10:40.494377 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11ffd92-2fbb-46fa-b40b-274937a24084 service placement] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:10:40.494966 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11ffd92-2fbb-46fa-b40b-274937a24084 service placement] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:10:40.495781 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11ffd92-2fbb-46fa-b40b-274937a24084 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:10:40.499455 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-e11ffd92-2fbb-46fa-b40b-274937a24084 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:10:40.523717 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11ffd92-2fbb-46fa-b40b-274937a24084 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:10:40.526833 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11ffd92-2fbb-46fa-b40b-274937a24084 service placement] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:10:40.550932 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1165/2321] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:10:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 14:10:44.158465 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fab344-107f-430b-8be0-3d5da2d9d4ab None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:10:44.159089 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fab344-107f-430b-8be0-3d5da2d9d4ab None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:10:44.159089 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fab344-107f-430b-8be0-3d5da2d9d4ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:10:44.211194 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-88fab344-107f-430b-8be0-3d5da2d9d4ab None None] MFA Rules not processed for user `1747d40a285942eb9b4d5bc14136d88a`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:10:44.212963 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-88fab344-107f-430b-8be0-3d5da2d9d4ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:10:44.359069 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1157/2322] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:10:44 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:10:44.834124 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11ffd92-2fbb-46fa-b40b-274937a24084 service placement] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:10:44.835619 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-e11ffd92-2fbb-46fa-b40b-274937a24084 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:10:44.937047 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11ffd92-2fbb-46fa-b40b-274937a24084 service placement] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:10:44.943007 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0762c6-6e9f-4857-813f-d8aec9395bae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:10:44.943989 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0762c6-6e9f-4857-813f-d8aec9395bae service neutron] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:10:44.944472 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0762c6-6e9f-4857-813f-d8aec9395bae service neutron] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:10:44.944941 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0762c6-6e9f-4857-813f-d8aec9395bae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:10:44.947809 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-9d0762c6-6e9f-4857-813f-d8aec9395bae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:10:44.961886 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0762c6-6e9f-4857-813f-d8aec9395bae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:10:44.963404 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0762c6-6e9f-4857-813f-d8aec9395bae service neutron] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:10:44.974914 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1166/2323] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:10:44 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 14:10:45.752760 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60164af8-e539-4cde-8326-088aa6ef1f84 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:10:45.753320 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60164af8-e539-4cde-8326-088aa6ef1f84 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:10:45.753320 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60164af8-e539-4cde-8326-088aa6ef1f84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:10:45.775278 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-60164af8-e539-4cde-8326-088aa6ef1f84 None None] MFA Rules not processed for user `56db6603f359477885b5151dec71c096`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:10:45.776510 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-60164af8-e539-4cde-8326-088aa6ef1f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:10:45.849381 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1158/2324] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:10:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:10:54.970299 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=58 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:10:55.841739 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=87 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:11:39.234911 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0762c6-6e9f-4857-813f-d8aec9395bae service neutron] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:11:39.244989 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0762c6-6e9f-4857-813f-d8aec9395bae service neutron] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:11:39.249768 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fef8160-0ef3-425c-84f0-6ec50b4b80ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:11:39.251040 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fef8160-0ef3-425c-84f0-6ec50b4b80ad service neutron] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:11:39.251040 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fef8160-0ef3-425c-84f0-6ec50b4b80ad service neutron] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:11:39.251343 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fef8160-0ef3-425c-84f0-6ec50b4b80ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:11:39.254015 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-4fef8160-0ef3-425c-84f0-6ec50b4b80ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:11:39.350179 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fef8160-0ef3-425c-84f0-6ec50b4b80ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:11:39.352135 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fef8160-0ef3-425c-84f0-6ec50b4b80ad service neutron] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:11:39.366774 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1167/2325] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:11:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:11:44.008583 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60164af8-e539-4cde-8326-088aa6ef1f84 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:11:44.013059 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-60164af8-e539-4cde-8326-088aa6ef1f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:11:44.032349 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60164af8-e539-4cde-8326-088aa6ef1f84 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:11:44.040782 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24f7a75-0830-4c42-b1fe-951d90d4cf76 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c69161cd520b4b29bf50f3bf2f1c033b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:11:44.041585 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24f7a75-0830-4c42-b1fe-951d90d4cf76 service nova] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:11:44.041858 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24f7a75-0830-4c42-b1fe-951d90d4cf76 service nova] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:11:44.042250 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24f7a75-0830-4c42-b1fe-951d90d4cf76 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:11:44.044192 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-c24f7a75-0830-4c42-b1fe-951d90d4cf76 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:11:44.088646 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24f7a75-0830-4c42-b1fe-951d90d4cf76 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:11:44.090519 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24f7a75-0830-4c42-b1fe-951d90d4cf76 service nova] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:11:44.103393 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1159/2326] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:11:44 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 14:11:44.601352 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fef8160-0ef3-425c-84f0-6ec50b4b80ad service neutron] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:11:44.614315 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fef8160-0ef3-425c-84f0-6ec50b4b80ad service neutron] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:11:44.621400 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dfbb5d-acf9-49ae-87e3-0544e88a0016 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c69161cd520b4b29bf50f3bf2f1c033b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:11:44.622221 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dfbb5d-acf9-49ae-87e3-0544e88a0016 service nova] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:11:44.622645 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dfbb5d-acf9-49ae-87e3-0544e88a0016 service nova] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:11:44.623092 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dfbb5d-acf9-49ae-87e3-0544e88a0016 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:11:44.628010 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-a0dfbb5d-acf9-49ae-87e3-0544e88a0016 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:11:44.644947 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dfbb5d-acf9-49ae-87e3-0544e88a0016 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:11:44.646722 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dfbb5d-acf9-49ae-87e3-0544e88a0016 service nova] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:11:44.657345 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1168/2327] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:11:44 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 14:11:54.096136 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=11 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:11:54.652800 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=40 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:12:41.280878 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24f7a75-0830-4c42-b1fe-951d90d4cf76 service nova] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:12:41.293994 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24f7a75-0830-4c42-b1fe-951d90d4cf76 service nova] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:12:41.299162 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa72c84-2ca1-49d8-9c42-42772e4544c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:12:41.299733 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa72c84-2ca1-49d8-9c42-42772e4544c3 service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:12:41.300019 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa72c84-2ca1-49d8-9c42-42772e4544c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:12:41.300295 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa72c84-2ca1-49d8-9c42-42772e4544c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:12:41.373970 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa72c84-2ca1-49d8-9c42-42772e4544c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:12:41.375489 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa72c84-2ca1-49d8-9c42-42772e4544c3 service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:12:41.393946 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1160/2328] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:12:41 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:12:41.526851 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dfbb5d-acf9-49ae-87e3-0544e88a0016 service nova] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:12:41.540739 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dfbb5d-acf9-49ae-87e3-0544e88a0016 service nova] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:12:41.545205 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af18b0ba-25d5-430c-bdd2-4606a5bfac68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:12:41.545911 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af18b0ba-25d5-430c-bdd2-4606a5bfac68 service cinder] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:12:41.546195 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af18b0ba-25d5-430c-bdd2-4606a5bfac68 service cinder] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:12:41.546553 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af18b0ba-25d5-430c-bdd2-4606a5bfac68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:12:41.548724 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-af18b0ba-25d5-430c-bdd2-4606a5bfac68 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:12:41.650477 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af18b0ba-25d5-430c-bdd2-4606a5bfac68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:12:41.651921 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af18b0ba-25d5-430c-bdd2-4606a5bfac68 service cinder] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:12:41.674370 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1169/2329] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:12:41 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:12:42.192888 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa72c84-2ca1-49d8-9c42-42772e4544c3 service cinder] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:12:42.266979 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa72c84-2ca1-49d8-9c42-42772e4544c3 service cinder] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:12:42.275332 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bf046d-e800-49c5-932e-b7418e94d39e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:12:42.275998 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bf046d-e800-49c5-932e-b7418e94d39e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:12:42.276285 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bf046d-e800-49c5-932e-b7418e94d39e admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:12:42.276567 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bf046d-e800-49c5-932e-b7418e94d39e admin admin] PATH_INFO: `/v3/users/df107ff94ddc4b7c889dfcdfeb746b47` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:12:42.277800 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bf046d-e800-49c5-932e-b7418e94d39e admin admin] RBAC: Authorizing `identity:delete_user(user_id=df107ff94ddc4b7c889dfcdfeb746b47)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:12:42.279757 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bf046d-e800-49c5-932e-b7418e94d39e admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:12:42.346876 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-02bf046d-e800-49c5-932e-b7418e94d39e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df107ff94ddc4b7c889dfcdfeb746b47'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:12:42.347628 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-02bf046d-e800-49c5-932e-b7418e94d39e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df107ff94ddc4b7c889dfcdfeb746b47'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:12:42.360078 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-02bf046d-e800-49c5-932e-b7418e94d39e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df107ff94ddc4b7c889dfcdfeb746b47'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:12:42.360078 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-02bf046d-e800-49c5-932e-b7418e94d39e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df107ff94ddc4b7c889dfcdfeb746b47'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:12:42.375227 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-02bf046d-e800-49c5-932e-b7418e94d39e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df107ff94ddc4b7c889dfcdfeb746b47'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:12:42.391137 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1161/2330] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 14:12:42 2022] DELETE /identity/v3/users/df107ff94ddc4b7c889dfcdfeb746b47 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:12:42.395763 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b983805e-ef30-4c65-aaac-fca936cf6562 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:12:42.396058 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b983805e-ef30-4c65-aaac-fca936cf6562 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:12:42.396648 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b983805e-ef30-4c65-aaac-fca936cf6562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:12:42.420119 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-b983805e-ef30-4c65-aaac-fca936cf6562 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:12:42.421379 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-b983805e-ef30-4c65-aaac-fca936cf6562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:12:42.507593 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1170/2331] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:12:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:12:42.520610 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bf046d-e800-49c5-932e-b7418e94d39e admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:12:42.523220 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-02bf046d-e800-49c5-932e-b7418e94d39e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:12:42.548568 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:12:42.620314 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bf046d-e800-49c5-932e-b7418e94d39e admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:12:42.624663 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15392b6f-2a8d-4692-a83c-7586588cf930 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:12:42.625232 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15392b6f-2a8d-4692-a83c-7586588cf930 service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:12:42.625533 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15392b6f-2a8d-4692-a83c-7586588cf930 service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:12:42.625864 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15392b6f-2a8d-4692-a83c-7586588cf930 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:12:42.635739 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15392b6f-2a8d-4692-a83c-7586588cf930 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:12:42.637182 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15392b6f-2a8d-4692-a83c-7586588cf930 service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:12:42.648290 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1162/2332] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:12:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:12:42.979984 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b983805e-ef30-4c65-aaac-fca936cf6562 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:12:42.982184 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-b983805e-ef30-4c65-aaac-fca936cf6562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:12:42.996825 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b983805e-ef30-4c65-aaac-fca936cf6562 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:12:43.002087 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37864e0c-f773-4112-8ad8-7e4f034a3eff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:12:43.002772 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37864e0c-f773-4112-8ad8-7e4f034a3eff admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:12:43.003056 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37864e0c-f773-4112-8ad8-7e4f034a3eff admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:12:43.003338 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37864e0c-f773-4112-8ad8-7e4f034a3eff admin admin] PATH_INFO: `/v3/projects/0307fce6884f440cba34f8db33fd956f` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:12:43.004749 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37864e0c-f773-4112-8ad8-7e4f034a3eff admin admin] RBAC: Authorizing `identity:delete_project(project_id=0307fce6884f440cba34f8db33fd956f)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:12:43.006729 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37864e0c-f773-4112-8ad8-7e4f034a3eff admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:12:43.077858 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-37864e0c-f773-4112-8ad8-7e4f034a3eff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0307fce6884f440cba34f8db33fd956f', 'request_id': 'req-37864e0c-f773-4112-8ad8-7e4f034a3eff'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:12:43.090730 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-37864e0c-f773-4112-8ad8-7e4f034a3eff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0307fce6884f440cba34f8db33fd956f', 'request_id': 'req-37864e0c-f773-4112-8ad8-7e4f034a3eff'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:12:43.099013 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-37864e0c-f773-4112-8ad8-7e4f034a3eff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0307fce6884f440cba34f8db33fd956f', 'request_id': 'req-37864e0c-f773-4112-8ad8-7e4f034a3eff'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:12:43.100464 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:12:43.119804 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-37864e0c-f773-4112-8ad8-7e4f034a3eff admin admin] The token cache is being invalidate because project 0307fce6884f440cba34f8db33fd956f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86933) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:12:43.120480 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-37864e0c-f773-4112-8ad8-7e4f034a3eff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:12:43.122092 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-37864e0c-f773-4112-8ad8-7e4f034a3eff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:12:43.123814 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1171/2333] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 14:12:42 2022] DELETE /identity/v3/projects/0307fce6884f440cba34f8db33fd956f => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:12:43.128854 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15392b6f-2a8d-4692-a83c-7586588cf930 service neutron] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:12:43.146224 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15392b6f-2a8d-4692-a83c-7586588cf930 service neutron] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:12:43.252353 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2ae0a5-01f4-439c-ac37-dafbea800c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:12:43.253443 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2ae0a5-01f4-439c-ac37-dafbea800c32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:12:43.253997 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2ae0a5-01f4-439c-ac37-dafbea800c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:12:43.255279 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2ae0a5-01f4-439c-ac37-dafbea800c32 admin admin] PATH_INFO: `/v3/users/f0cf3bb2c3f946938f1dbfd42feae40e` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:12:43.256927 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2ae0a5-01f4-439c-ac37-dafbea800c32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0cf3bb2c3f946938f1dbfd42feae40e)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:12:43.258903 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2ae0a5-01f4-439c-ac37-dafbea800c32 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:12:43.329807 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-3b2ae0a5-01f4-439c-ac37-dafbea800c32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0cf3bb2c3f946938f1dbfd42feae40e'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:12:43.330514 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-3b2ae0a5-01f4-439c-ac37-dafbea800c32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0cf3bb2c3f946938f1dbfd42feae40e'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:12:43.338317 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-3b2ae0a5-01f4-439c-ac37-dafbea800c32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0cf3bb2c3f946938f1dbfd42feae40e'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:12:43.339037 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-3b2ae0a5-01f4-439c-ac37-dafbea800c32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0cf3bb2c3f946938f1dbfd42feae40e'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:12:43.361270 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-3b2ae0a5-01f4-439c-ac37-dafbea800c32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0cf3bb2c3f946938f1dbfd42feae40e'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:12:43.380295 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1163/2334] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 14:12:43 2022] DELETE /identity/v3/users/f0cf3bb2c3f946938f1dbfd42feae40e => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:12:43.718859 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37864e0c-f773-4112-8ad8-7e4f034a3eff admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:12:43.721809 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-37864e0c-f773-4112-8ad8-7e4f034a3eff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:12:43.739591 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37864e0c-f773-4112-8ad8-7e4f034a3eff admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:12:43.821119 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafb37db-7df8-4c64-aaf8-ecf91ff6332a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:12:43.821868 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafb37db-7df8-4c64-aaf8-ecf91ff6332a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:12:43.822153 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafb37db-7df8-4c64-aaf8-ecf91ff6332a admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:12:43.822442 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafb37db-7df8-4c64-aaf8-ecf91ff6332a admin admin] PATH_INFO: `/v3/projects/2d158719c484419e857674c8877b83ca` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:12:43.823724 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafb37db-7df8-4c64-aaf8-ecf91ff6332a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d158719c484419e857674c8877b83ca)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:12:43.825967 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafb37db-7df8-4c64-aaf8-ecf91ff6332a admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:12:43.880125 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-eafb37db-7df8-4c64-aaf8-ecf91ff6332a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d158719c484419e857674c8877b83ca', 'request_id': 'req-eafb37db-7df8-4c64-aaf8-ecf91ff6332a'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:12:43.891715 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-eafb37db-7df8-4c64-aaf8-ecf91ff6332a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d158719c484419e857674c8877b83ca', 'request_id': 'req-eafb37db-7df8-4c64-aaf8-ecf91ff6332a'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:12:43.905945 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-eafb37db-7df8-4c64-aaf8-ecf91ff6332a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d158719c484419e857674c8877b83ca', 'request_id': 'req-eafb37db-7df8-4c64-aaf8-ecf91ff6332a'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:12:43.919479 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-eafb37db-7df8-4c64-aaf8-ecf91ff6332a admin admin] The token cache is being invalidate because project 2d158719c484419e857674c8877b83ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86933) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:12:43.919775 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-eafb37db-7df8-4c64-aaf8-ecf91ff6332a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:12:43.920444 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-eafb37db-7df8-4c64-aaf8-ecf91ff6332a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:12:43.922073 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1172/2335] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 14:12:43 2022] DELETE /identity/v3/projects/2d158719c484419e857674c8877b83ca => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:12:43.928969 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2530a0-3c4c-4d36-92a3-43e8698ba621 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:12:43.929460 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2530a0-3c4c-4d36-92a3-43e8698ba621 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:12:43.929875 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2530a0-3c4c-4d36-92a3-43e8698ba621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:12:43.952868 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-2e2530a0-3c4c-4d36-92a3-43e8698ba621 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:12:43.954176 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-2e2530a0-3c4c-4d36-92a3-43e8698ba621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:12:43.989432 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:12:44.047479 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1164/2336] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:12:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:12:44.060606 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0abb30-e209-4bf9-b8b8-5309e861a18e None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:12:44.061198 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0abb30-e209-4bf9-b8b8-5309e861a18e None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:12:44.061198 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0abb30-e209-4bf9-b8b8-5309e861a18e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:12:44.084901 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-5a0abb30-e209-4bf9-b8b8-5309e861a18e None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:12:44.086181 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-5a0abb30-e209-4bf9-b8b8-5309e861a18e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:12:44.095182 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1173/2337] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:12:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:12:44.099979 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2530a0-3c4c-4d36-92a3-43e8698ba621 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:12:44.111763 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2530a0-3c4c-4d36-92a3-43e8698ba621 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:12:44.116955 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cb0309-af8a-44e1-a02f-107343f9612a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:12:44.117625 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cb0309-af8a-44e1-a02f-107343f9612a admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:12:44.117915 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cb0309-af8a-44e1-a02f-107343f9612a admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:12:44.118232 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cb0309-af8a-44e1-a02f-107343f9612a admin admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:12:44.118957 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cb0309-af8a-44e1-a02f-107343f9612a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:12:44.119262 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cb0309-af8a-44e1-a02f-107343f9612a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:12:44.120902 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cb0309-af8a-44e1-a02f-107343f9612a admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:12:44.136099 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1165/2338] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Jul 26 14:12:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:12:44.140852 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0abb30-e209-4bf9-b8b8-5309e861a18e None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:12:44.151776 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0abb30-e209-4bf9-b8b8-5309e861a18e None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:12:44.157624 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c172b7-f6be-4af9-adfc-8364c6fc6417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:12:44.158481 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c172b7-f6be-4af9-adfc-8364c6fc6417 admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:12:44.158855 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c172b7-f6be-4af9-adfc-8364c6fc6417 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:12:44.159162 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c172b7-f6be-4af9-adfc-8364c6fc6417 admin admin] PATH_INFO: `/v3/projects` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:12:44.160023 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c172b7-f6be-4af9-adfc-8364c6fc6417 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:12:44.163286 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c172b7-f6be-4af9-adfc-8364c6fc6417 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:12:44.217696 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1174/2339] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 14:12:44 2022] POST /identity/v3/projects => generated 367 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:12:44.220948 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cb0309-af8a-44e1-a02f-107343f9612a admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:12:44.240306 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cb0309-af8a-44e1-a02f-107343f9612a admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:12:44.365768 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4efc596-0bf6-4e9b-b2bc-58812fb2f819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:12:44.367925 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4efc596-0bf6-4e9b-b2bc-58812fb2f819 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:12:44.368845 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4efc596-0bf6-4e9b-b2bc-58812fb2f819 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:12:44.369792 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4efc596-0bf6-4e9b-b2bc-58812fb2f819 admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:12:44.370738 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4efc596-0bf6-4e9b-b2bc-58812fb2f819 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:12:44.374992 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4efc596-0bf6-4e9b-b2bc-58812fb2f819 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:12:44.403043 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1166/2340] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 14:12:44 2022] POST /identity/v3/users => generated 284 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 14:12:44.408010 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c172b7-f6be-4af9-adfc-8364c6fc6417 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:12:44.420818 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c172b7-f6be-4af9-adfc-8364c6fc6417 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:12:44.425620 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cc212a-ade0-4c8f-89ef-c5cafd81b4b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:12:44.426258 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc212a-ade0-4c8f-89ef-c5cafd81b4b6 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:12:44.426543 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc212a-ade0-4c8f-89ef-c5cafd81b4b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:12:44.426819 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc212a-ade0-4c8f-89ef-c5cafd81b4b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:12:44.427531 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cc212a-ade0-4c8f-89ef-c5cafd81b4b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:12:44.429144 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cc212a-ade0-4c8f-89ef-c5cafd81b4b6 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:12:44.441903 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1175/2341] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:12:44 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:12:44.446082 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4efc596-0bf6-4e9b-b2bc-58812fb2f819 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:12:44.460200 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4efc596-0bf6-4e9b-b2bc-58812fb2f819 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:12:44.467643 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ac3025-efbc-40e7-ab9b-64d5097ff8f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:12:44.470384 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ac3025-efbc-40e7-ab9b-64d5097ff8f4 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:12:44.470384 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ac3025-efbc-40e7-ab9b-64d5097ff8f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:12:44.470384 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ac3025-efbc-40e7-ab9b-64d5097ff8f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:12:44.470384 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ac3025-efbc-40e7-ab9b-64d5097ff8f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:12:44.471463 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ac3025-efbc-40e7-ab9b-64d5097ff8f4 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:12:44.483199 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1167/2342] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:12:44 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:12:44.491127 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cc212a-ade0-4c8f-89ef-c5cafd81b4b6 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:12:44.509166 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cc212a-ade0-4c8f-89ef-c5cafd81b4b6 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:12:44.515082 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997cbabe-07c2-4886-9b2c-a80132e92ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:12:44.515774 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997cbabe-07c2-4886-9b2c-a80132e92ecb admin admin] REQUEST_METHOD: `PUT` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:12:44.516125 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997cbabe-07c2-4886-9b2c-a80132e92ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:12:44.516401 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997cbabe-07c2-4886-9b2c-a80132e92ecb admin admin] PATH_INFO: `/v3/projects/79e5a014a42a47799916824962147b58/users/0a96347d110440799d55da81cf73a3dc/roles/759f389b221049c4b01faf3b984cd8d7` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:12:44.550241 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997cbabe-07c2-4886-9b2c-a80132e92ecb admin admin] RBAC: Authorizing `identity:create_grant(project_id=79e5a014a42a47799916824962147b58, user_id=0a96347d110440799d55da81cf73a3dc, role_id=759f389b221049c4b01faf3b984cd8d7)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:12:44.553096 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997cbabe-07c2-4886-9b2c-a80132e92ecb admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:12:44.571651 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1176/2343] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 14:12:44 2022] PUT /identity/v3/projects/79e5a014a42a47799916824962147b58/users/0a96347d110440799d55da81cf73a3dc/roles/759f389b221049c4b01faf3b984cd8d7 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:12:44.653627 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bad9d15-fbf6-4331-b441-bee6a2d4cde6 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:12:44.653913 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bad9d15-fbf6-4331-b441-bee6a2d4cde6 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:12:44.654196 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bad9d15-fbf6-4331-b441-bee6a2d4cde6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:12:44.694277 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-4bad9d15-fbf6-4331-b441-bee6a2d4cde6 None None] MFA Rules not processed for user `0a96347d110440799d55da81cf73a3dc`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:12:44.695528 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-4bad9d15-fbf6-4331-b441-bee6a2d4cde6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:12:44.784996 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1168/2344] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:12:44 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:12:44.791809 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997cbabe-07c2-4886-9b2c-a80132e92ecb admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:12:44.814464 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:12:44.871998 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997cbabe-07c2-4886-9b2c-a80132e92ecb admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:12:44.877801 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74607393-8eb3-43ba-b15f-7fc3022fd547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:12:44.878476 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74607393-8eb3-43ba-b15f-7fc3022fd547 admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:12:44.878754 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74607393-8eb3-43ba-b15f-7fc3022fd547 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:12:44.879054 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74607393-8eb3-43ba-b15f-7fc3022fd547 admin admin] PATH_INFO: `/v3/projects` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:12:44.879759 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74607393-8eb3-43ba-b15f-7fc3022fd547 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:12:44.881733 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74607393-8eb3-43ba-b15f-7fc3022fd547 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:12:44.918780 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1177/2345] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 14:12:44 2022] POST /identity/v3/projects => generated 367 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 14:12:44.923930 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bad9d15-fbf6-4331-b441-bee6a2d4cde6 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:12:44.936432 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bad9d15-fbf6-4331-b441-bee6a2d4cde6 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:12:44.963853 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:12:45.009277 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac78fbf-e18d-40bb-8cf6-8491d2b7f83b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:12:45.009971 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac78fbf-e18d-40bb-8cf6-8491d2b7f83b admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:12:45.010252 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac78fbf-e18d-40bb-8cf6-8491d2b7f83b admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:12:45.010531 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac78fbf-e18d-40bb-8cf6-8491d2b7f83b admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:12:45.011218 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac78fbf-e18d-40bb-8cf6-8491d2b7f83b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:12:45.013206 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac78fbf-e18d-40bb-8cf6-8491d2b7f83b admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:12:45.044559 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1169/2346] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 14:12:44 2022] POST /identity/v3/users => generated 290 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 14:12:45.050003 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74607393-8eb3-43ba-b15f-7fc3022fd547 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:12:45.065369 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74607393-8eb3-43ba-b15f-7fc3022fd547 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:12:45.072881 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d0d3c7-a8f0-40e2-9439-043c46014553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:12:45.073928 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d0d3c7-a8f0-40e2-9439-043c46014553 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:12:45.074290 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d0d3c7-a8f0-40e2-9439-043c46014553 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:12:45.074739 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d0d3c7-a8f0-40e2-9439-043c46014553 admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:12:45.075923 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d0d3c7-a8f0-40e2-9439-043c46014553 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:12:45.078070 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d0d3c7-a8f0-40e2-9439-043c46014553 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:12:45.090940 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1178/2347] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:12:45 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:12:45.096189 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac78fbf-e18d-40bb-8cf6-8491d2b7f83b admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:12:45.108310 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac78fbf-e18d-40bb-8cf6-8491d2b7f83b admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:12:45.113234 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db1eabf-8173-4575-8ccf-66ae691fcedb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:12:45.113945 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db1eabf-8173-4575-8ccf-66ae691fcedb admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:12:45.114383 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db1eabf-8173-4575-8ccf-66ae691fcedb admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:12:45.114665 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db1eabf-8173-4575-8ccf-66ae691fcedb admin admin] PATH_INFO: `/v3/projects/655f3d6aa32c49168d96fe9973204658/users/4d7bf2cc881342cc8b0e635c4a796e31/roles/6e0201a9a6484f7d8b112b1eafff70bd` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:12:45.133565 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db1eabf-8173-4575-8ccf-66ae691fcedb admin admin] RBAC: Authorizing `identity:create_grant(project_id=655f3d6aa32c49168d96fe9973204658, user_id=4d7bf2cc881342cc8b0e635c4a796e31, role_id=6e0201a9a6484f7d8b112b1eafff70bd)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:12:45.138891 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db1eabf-8173-4575-8ccf-66ae691fcedb admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:12:45.155019 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1170/2348] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 14:12:45 2022] PUT /identity/v3/projects/655f3d6aa32c49168d96fe9973204658/users/4d7bf2cc881342cc8b0e635c4a796e31/roles/6e0201a9a6484f7d8b112b1eafff70bd => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:12:45.161298 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d0d3c7-a8f0-40e2-9439-043c46014553 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:12:45.180280 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d0d3c7-a8f0-40e2-9439-043c46014553 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:12:45.264008 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afda5e44-fab1-41f7-bb25-400655c168f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:12:45.264632 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afda5e44-fab1-41f7-bb25-400655c168f3 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:12:45.264946 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afda5e44-fab1-41f7-bb25-400655c168f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:12:45.265218 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afda5e44-fab1-41f7-bb25-400655c168f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:12:45.265962 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afda5e44-fab1-41f7-bb25-400655c168f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:12:45.267711 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afda5e44-fab1-41f7-bb25-400655c168f3 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:12:45.278778 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1179/2349] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:12:45 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:12:45.283191 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db1eabf-8173-4575-8ccf-66ae691fcedb admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:12:45.295359 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db1eabf-8173-4575-8ccf-66ae691fcedb admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:12:45.302694 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b7399e-4897-47c4-b31e-1dd252c2e124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:12:45.303380 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b7399e-4897-47c4-b31e-1dd252c2e124 admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:12:45.303665 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b7399e-4897-47c4-b31e-1dd252c2e124 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:12:45.303945 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b7399e-4897-47c4-b31e-1dd252c2e124 admin admin] PATH_INFO: `/v3/projects/655f3d6aa32c49168d96fe9973204658/users/4d7bf2cc881342cc8b0e635c4a796e31/roles/6e0201a9a6484f7d8b112b1eafff70bd` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:12:45.306484 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b7399e-4897-47c4-b31e-1dd252c2e124 admin admin] RBAC: Authorizing `identity:create_grant(project_id=655f3d6aa32c49168d96fe9973204658, user_id=4d7bf2cc881342cc8b0e635c4a796e31, role_id=6e0201a9a6484f7d8b112b1eafff70bd)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:12:45.309131 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b7399e-4897-47c4-b31e-1dd252c2e124 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:12:45.320568 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1171/2350] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 14:12:45 2022] PUT /identity/v3/projects/655f3d6aa32c49168d96fe9973204658/users/4d7bf2cc881342cc8b0e635c4a796e31/roles/6e0201a9a6484f7d8b112b1eafff70bd => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:12:45.331942 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9819faea-dc1b-4b1f-bd3c-04789f5681f2 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:12:45.332238 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9819faea-dc1b-4b1f-bd3c-04789f5681f2 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:12:45.332523 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9819faea-dc1b-4b1f-bd3c-04789f5681f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:12:45.363201 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-9819faea-dc1b-4b1f-bd3c-04789f5681f2 None None] MFA Rules not processed for user `4d7bf2cc881342cc8b0e635c4a796e31`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:12:45.364530 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-9819faea-dc1b-4b1f-bd3c-04789f5681f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:12:45.441308 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1180/2351] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:12:45 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:12:55.316223 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=33,keystone:INSERT=5 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:12:55.436421 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=92,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:13:39.854128 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b7399e-4897-47c4-b31e-1dd252c2e124 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:13:39.855895 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-c9b7399e-4897-47c4-b31e-1dd252c2e124 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:13:39.938507 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b7399e-4897-47c4-b31e-1dd252c2e124 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:13:39.944284 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac1293e-5679-4749-84e4-d6f1e2a71ae5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c69161cd520b4b29bf50f3bf2f1c033b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:13:39.944844 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac1293e-5679-4749-84e4-d6f1e2a71ae5 service nova] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:13:39.945658 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac1293e-5679-4749-84e4-d6f1e2a71ae5 service nova] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:13:39.945931 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac1293e-5679-4749-84e4-d6f1e2a71ae5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:13:39.948242 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-5ac1293e-5679-4749-84e4-d6f1e2a71ae5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:13:40.040426 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac1293e-5679-4749-84e4-d6f1e2a71ae5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:13:40.042135 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac1293e-5679-4749-84e4-d6f1e2a71ae5 service nova] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:13:40.051883 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1172/2352] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:13:39 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:13:43.944889 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9819faea-dc1b-4b1f-bd3c-04789f5681f2 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:13:43.946407 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-9819faea-dc1b-4b1f-bd3c-04789f5681f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:13:44.006749 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9819faea-dc1b-4b1f-bd3c-04789f5681f2 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:13:44.010492 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ba6f9b-a2e0-43da-8ed4-46cd0171f77a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:13:44.011126 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ba6f9b-a2e0-43da-8ed4-46cd0171f77a service neutron] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:13:44.011533 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ba6f9b-a2e0-43da-8ed4-46cd0171f77a service neutron] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:13:44.011892 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ba6f9b-a2e0-43da-8ed4-46cd0171f77a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:13:44.013637 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-76ba6f9b-a2e0-43da-8ed4-46cd0171f77a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:13:44.028898 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ba6f9b-a2e0-43da-8ed4-46cd0171f77a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:13:44.030397 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ba6f9b-a2e0-43da-8ed4-46cd0171f77a service neutron] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:13:44.039866 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1181/2353] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:13:43 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:13:45.485106 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac1293e-5679-4749-84e4-d6f1e2a71ae5 service nova] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:13:45.561452 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac1293e-5679-4749-84e4-d6f1e2a71ae5 service nova] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:13:45.567371 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff8944b-9ee7-4af5-9443-5e15e00e8e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:13:45.568044 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff8944b-9ee7-4af5-9443-5e15e00e8e6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:13:45.568361 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff8944b-9ee7-4af5-9443-5e15e00e8e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:13:45.568639 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff8944b-9ee7-4af5-9443-5e15e00e8e6d admin admin] PATH_INFO: `/v3/users/0a96347d110440799d55da81cf73a3dc` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:13:45.569912 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff8944b-9ee7-4af5-9443-5e15e00e8e6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a96347d110440799d55da81cf73a3dc)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:13:45.572663 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff8944b-9ee7-4af5-9443-5e15e00e8e6d admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:13:45.617092 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:13:45.647246 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-aff8944b-9ee7-4af5-9443-5e15e00e8e6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a96347d110440799d55da81cf73a3dc'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:13:45.647924 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-aff8944b-9ee7-4af5-9443-5e15e00e8e6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a96347d110440799d55da81cf73a3dc'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:13:45.656062 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-aff8944b-9ee7-4af5-9443-5e15e00e8e6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a96347d110440799d55da81cf73a3dc'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:13:45.656770 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-aff8944b-9ee7-4af5-9443-5e15e00e8e6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a96347d110440799d55da81cf73a3dc'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:13:45.669521 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-aff8944b-9ee7-4af5-9443-5e15e00e8e6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a96347d110440799d55da81cf73a3dc'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:13:45.702022 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1173/2354] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 14:13:45 2022] DELETE /identity/v3/users/0a96347d110440799d55da81cf73a3dc => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:13:45.706564 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6cfa0c-ec67-4c65-8072-8b5cd9ca332f None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:13:45.707149 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6cfa0c-ec67-4c65-8072-8b5cd9ca332f None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:13:45.707149 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6cfa0c-ec67-4c65-8072-8b5cd9ca332f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:13:45.728523 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-cd6cfa0c-ec67-4c65-8072-8b5cd9ca332f None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:13:45.730425 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-cd6cfa0c-ec67-4c65-8072-8b5cd9ca332f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:13:45.797097 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1182/2355] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:13:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:13:45.807548 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff8944b-9ee7-4af5-9443-5e15e00e8e6d admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:13:45.809063 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-aff8944b-9ee7-4af5-9443-5e15e00e8e6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:13:45.880400 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff8944b-9ee7-4af5-9443-5e15e00e8e6d admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:13:45.884891 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa11f254-8334-4dca-a7a4-e12ed8a2321d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:13:45.885549 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa11f254-8334-4dca-a7a4-e12ed8a2321d service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:13:45.885833 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa11f254-8334-4dca-a7a4-e12ed8a2321d service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:13:45.886108 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa11f254-8334-4dca-a7a4-e12ed8a2321d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:13:45.896550 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa11f254-8334-4dca-a7a4-e12ed8a2321d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:13:45.898043 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa11f254-8334-4dca-a7a4-e12ed8a2321d service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:13:45.909703 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1174/2356] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:13:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:13:46.242205 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6cfa0c-ec67-4c65-8072-8b5cd9ca332f None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:13:46.243919 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-cd6cfa0c-ec67-4c65-8072-8b5cd9ca332f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:13:46.266817 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6cfa0c-ec67-4c65-8072-8b5cd9ca332f None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:13:46.275396 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdee3e7-3778-420f-bacb-7e1b1f900229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:13:46.277394 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdee3e7-3778-420f-bacb-7e1b1f900229 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:13:46.277920 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdee3e7-3778-420f-bacb-7e1b1f900229 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:13:46.278223 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdee3e7-3778-420f-bacb-7e1b1f900229 admin admin] PATH_INFO: `/v3/projects/79e5a014a42a47799916824962147b58` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:13:46.279537 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdee3e7-3778-420f-bacb-7e1b1f900229 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79e5a014a42a47799916824962147b58)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:13:46.282225 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdee3e7-3778-420f-bacb-7e1b1f900229 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:13:46.351370 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-6cdee3e7-3778-420f-bacb-7e1b1f900229 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79e5a014a42a47799916824962147b58', 'request_id': 'req-6cdee3e7-3778-420f-bacb-7e1b1f900229'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:13:46.361650 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-6cdee3e7-3778-420f-bacb-7e1b1f900229 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79e5a014a42a47799916824962147b58', 'request_id': 'req-6cdee3e7-3778-420f-bacb-7e1b1f900229'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:13:46.367876 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-6cdee3e7-3778-420f-bacb-7e1b1f900229 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79e5a014a42a47799916824962147b58', 'request_id': 'req-6cdee3e7-3778-420f-bacb-7e1b1f900229'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:13:46.385851 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-6cdee3e7-3778-420f-bacb-7e1b1f900229 admin admin] The token cache is being invalidate because project 79e5a014a42a47799916824962147b58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86933) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:13:46.386310 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-6cdee3e7-3778-420f-bacb-7e1b1f900229 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:13:46.387091 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-6cdee3e7-3778-420f-bacb-7e1b1f900229 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:13:46.388702 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1183/2357] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 14:13:46 2022] DELETE /identity/v3/projects/79e5a014a42a47799916824962147b58 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:13:46.392851 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa11f254-8334-4dca-a7a4-e12ed8a2321d service neutron] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:13:46.403645 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa11f254-8334-4dca-a7a4-e12ed8a2321d service neutron] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:13:46.476622 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a18fb8-ab4b-40ef-a00e-91a07f4deca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:13:46.477634 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a18fb8-ab4b-40ef-a00e-91a07f4deca7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:13:46.478114 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a18fb8-ab4b-40ef-a00e-91a07f4deca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:13:46.478543 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a18fb8-ab4b-40ef-a00e-91a07f4deca7 admin admin] PATH_INFO: `/v3/users/4d7bf2cc881342cc8b0e635c4a796e31` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:13:46.482751 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a18fb8-ab4b-40ef-a00e-91a07f4deca7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d7bf2cc881342cc8b0e635c4a796e31)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:13:46.485624 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a18fb8-ab4b-40ef-a00e-91a07f4deca7 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:13:46.558764 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-c6a18fb8-ab4b-40ef-a00e-91a07f4deca7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d7bf2cc881342cc8b0e635c4a796e31'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:13:46.559596 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-c6a18fb8-ab4b-40ef-a00e-91a07f4deca7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d7bf2cc881342cc8b0e635c4a796e31'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:13:46.562425 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:13:46.569670 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-c6a18fb8-ab4b-40ef-a00e-91a07f4deca7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d7bf2cc881342cc8b0e635c4a796e31'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:13:46.571037 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-c6a18fb8-ab4b-40ef-a00e-91a07f4deca7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d7bf2cc881342cc8b0e635c4a796e31'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:13:46.589836 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-c6a18fb8-ab4b-40ef-a00e-91a07f4deca7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d7bf2cc881342cc8b0e635c4a796e31'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:13:46.605016 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1175/2358] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 14:13:46 2022] DELETE /identity/v3/users/4d7bf2cc881342cc8b0e635c4a796e31 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:13:46.937493 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdee3e7-3778-420f-bacb-7e1b1f900229 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:13:46.938953 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-6cdee3e7-3778-420f-bacb-7e1b1f900229 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:13:46.954860 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdee3e7-3778-420f-bacb-7e1b1f900229 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:13:46.987431 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:13:47.029080 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168f8188-97be-447d-aff4-bb49e787c2fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:13:47.029796 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168f8188-97be-447d-aff4-bb49e787c2fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:13:47.030104 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168f8188-97be-447d-aff4-bb49e787c2fb admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:13:47.030357 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168f8188-97be-447d-aff4-bb49e787c2fb admin admin] PATH_INFO: `/v3/projects/655f3d6aa32c49168d96fe9973204658` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:13:47.035710 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168f8188-97be-447d-aff4-bb49e787c2fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=655f3d6aa32c49168d96fe9973204658)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:13:47.035710 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168f8188-97be-447d-aff4-bb49e787c2fb admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:13:47.093711 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-168f8188-97be-447d-aff4-bb49e787c2fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '655f3d6aa32c49168d96fe9973204658', 'request_id': 'req-168f8188-97be-447d-aff4-bb49e787c2fb'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:13:47.103352 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-168f8188-97be-447d-aff4-bb49e787c2fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '655f3d6aa32c49168d96fe9973204658', 'request_id': 'req-168f8188-97be-447d-aff4-bb49e787c2fb'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:13:47.110336 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-168f8188-97be-447d-aff4-bb49e787c2fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '655f3d6aa32c49168d96fe9973204658', 'request_id': 'req-168f8188-97be-447d-aff4-bb49e787c2fb'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:13:47.124403 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-168f8188-97be-447d-aff4-bb49e787c2fb admin admin] The token cache is being invalidate because project 655f3d6aa32c49168d96fe9973204658 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86933) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:13:47.124763 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-168f8188-97be-447d-aff4-bb49e787c2fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:13:47.125417 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-168f8188-97be-447d-aff4-bb49e787c2fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:13:47.126909 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1184/2359] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 14:13:46 2022] DELETE /identity/v3/projects/655f3d6aa32c49168d96fe9973204658 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:13:47.135771 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7932a7e0-4bed-42a6-8a24-1ea491327edb None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:13:47.136193 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7932a7e0-4bed-42a6-8a24-1ea491327edb None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:13:47.136512 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7932a7e0-4bed-42a6-8a24-1ea491327edb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:13:47.157745 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-7932a7e0-4bed-42a6-8a24-1ea491327edb None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:13:47.159000 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-7932a7e0-4bed-42a6-8a24-1ea491327edb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:13:47.231924 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1176/2360] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:13:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:13:47.239673 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328f663a-781e-4b9f-b2c6-168067074b5b None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:13:47.239963 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328f663a-781e-4b9f-b2c6-168067074b5b None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:13:47.240238 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328f663a-781e-4b9f-b2c6-168067074b5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:13:47.263936 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-328f663a-781e-4b9f-b2c6-168067074b5b None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:13:47.265169 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-328f663a-781e-4b9f-b2c6-168067074b5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:13:47.274055 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1185/2361] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:13:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:13:47.283527 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7932a7e0-4bed-42a6-8a24-1ea491327edb None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:13:47.290079 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7932a7e0-4bed-42a6-8a24-1ea491327edb None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:13:47.295811 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb0839c-0a57-494b-90d8-64b125e77401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:13:47.296441 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb0839c-0a57-494b-90d8-64b125e77401 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:13:47.296727 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb0839c-0a57-494b-90d8-64b125e77401 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:13:47.297004 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb0839c-0a57-494b-90d8-64b125e77401 admin admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:13:47.297745 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb0839c-0a57-494b-90d8-64b125e77401 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:13:47.298047 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb0839c-0a57-494b-90d8-64b125e77401 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:13:47.299680 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb0839c-0a57-494b-90d8-64b125e77401 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:13:47.318103 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1177/2362] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Jul 26 14:13:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:13:47.323341 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328f663a-781e-4b9f-b2c6-168067074b5b None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:13:47.335754 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328f663a-781e-4b9f-b2c6-168067074b5b None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:13:47.338497 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b1d8e8-9e03-4460-a344-d85ee8342cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:13:47.339137 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b1d8e8-9e03-4460-a344-d85ee8342cf8 admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:13:47.339420 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b1d8e8-9e03-4460-a344-d85ee8342cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:13:47.339698 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b1d8e8-9e03-4460-a344-d85ee8342cf8 admin admin] PATH_INFO: `/v3/projects` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:13:47.340378 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b1d8e8-9e03-4460-a344-d85ee8342cf8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:13:47.342576 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b1d8e8-9e03-4460-a344-d85ee8342cf8 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:13:47.383336 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1186/2363] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 14:13:47 2022] POST /identity/v3/projects => generated 399 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 14:13:47.384560 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb0839c-0a57-494b-90d8-64b125e77401 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:13:47.397017 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb0839c-0a57-494b-90d8-64b125e77401 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:13:47.468875 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb1e80c-891c-4846-bd4d-f15befda8148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:13:47.469571 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb1e80c-891c-4846-bd4d-f15befda8148 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:13:47.469974 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb1e80c-891c-4846-bd4d-f15befda8148 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:13:47.470237 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb1e80c-891c-4846-bd4d-f15befda8148 admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:13:47.470893 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb1e80c-891c-4846-bd4d-f15befda8148 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:13:47.472998 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb1e80c-891c-4846-bd4d-f15befda8148 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:13:47.496105 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1178/2364] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 14:13:47 2022] POST /identity/v3/users => generated 300 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:13:47.500755 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b1d8e8-9e03-4460-a344-d85ee8342cf8 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:13:47.511795 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b1d8e8-9e03-4460-a344-d85ee8342cf8 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:13:47.516658 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d02e38c-360e-4d96-b19b-43693cbdaacd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:13:47.517285 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d02e38c-360e-4d96-b19b-43693cbdaacd admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:13:47.517611 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d02e38c-360e-4d96-b19b-43693cbdaacd admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:13:47.517889 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d02e38c-360e-4d96-b19b-43693cbdaacd admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:13:47.518598 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d02e38c-360e-4d96-b19b-43693cbdaacd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:13:47.520195 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d02e38c-360e-4d96-b19b-43693cbdaacd admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:13:47.532135 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1187/2365] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:13:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:13:47.537242 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb1e80c-891c-4846-bd4d-f15befda8148 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:13:47.553394 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb1e80c-891c-4846-bd4d-f15befda8148 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:13:47.560467 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a21c40-dbdd-4fa9-8c31-3f0c3e673bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:13:47.561468 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a21c40-dbdd-4fa9-8c31-3f0c3e673bd7 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:13:47.561971 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a21c40-dbdd-4fa9-8c31-3f0c3e673bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:13:47.562526 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a21c40-dbdd-4fa9-8c31-3f0c3e673bd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:13:47.563621 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a21c40-dbdd-4fa9-8c31-3f0c3e673bd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:13:47.566502 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a21c40-dbdd-4fa9-8c31-3f0c3e673bd7 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:13:47.581444 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1179/2366] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:13:47 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:13:47.585743 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d02e38c-360e-4d96-b19b-43693cbdaacd admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:13:47.596454 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d02e38c-360e-4d96-b19b-43693cbdaacd admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:13:47.601135 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862aac47-397a-4dd1-9008-03a776e32107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:13:47.601844 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862aac47-397a-4dd1-9008-03a776e32107 admin admin] REQUEST_METHOD: `PUT` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:13:47.602125 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862aac47-397a-4dd1-9008-03a776e32107 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:13:47.602411 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862aac47-397a-4dd1-9008-03a776e32107 admin admin] PATH_INFO: `/v3/projects/48c5bfd0920f4ffe85dbdce45eb4723a/users/86a2359178c04c70817c04d91eb6e986/roles/759f389b221049c4b01faf3b984cd8d7` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:13:47.618471 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862aac47-397a-4dd1-9008-03a776e32107 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48c5bfd0920f4ffe85dbdce45eb4723a, user_id=86a2359178c04c70817c04d91eb6e986, role_id=759f389b221049c4b01faf3b984cd8d7)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:13:47.621117 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862aac47-397a-4dd1-9008-03a776e32107 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:13:47.636759 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1188/2367] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 14:13:47 2022] PUT /identity/v3/projects/48c5bfd0920f4ffe85dbdce45eb4723a/users/86a2359178c04c70817c04d91eb6e986/roles/759f389b221049c4b01faf3b984cd8d7 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:13:47.648780 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5370a353-35e8-454c-8518-4317e00bb971 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:13:47.649071 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5370a353-35e8-454c-8518-4317e00bb971 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:13:47.649384 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5370a353-35e8-454c-8518-4317e00bb971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:13:47.679595 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-5370a353-35e8-454c-8518-4317e00bb971 None None] MFA Rules not processed for user `86a2359178c04c70817c04d91eb6e986`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:13:47.680862 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-5370a353-35e8-454c-8518-4317e00bb971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:13:47.704087 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:13:47.763381 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1180/2368] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:13:47 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:13:47.768767 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862aac47-397a-4dd1-9008-03a776e32107 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:13:47.827832 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:13:47.842192 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862aac47-397a-4dd1-9008-03a776e32107 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:13:47.850178 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e402e99f-7c56-48dd-912c-857ebcb8649c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:13:47.850821 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e402e99f-7c56-48dd-912c-857ebcb8649c admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:13:47.851138 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e402e99f-7c56-48dd-912c-857ebcb8649c admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:13:47.851413 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e402e99f-7c56-48dd-912c-857ebcb8649c admin admin] PATH_INFO: `/v3/projects` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:13:47.852121 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e402e99f-7c56-48dd-912c-857ebcb8649c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:13:47.854681 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e402e99f-7c56-48dd-912c-857ebcb8649c admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:13:47.884313 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1189/2369] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 14:13:47 2022] POST /identity/v3/projects => generated 401 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:13:47.888538 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5370a353-35e8-454c-8518-4317e00bb971 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:13:47.900104 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5370a353-35e8-454c-8518-4317e00bb971 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:13:47.964210 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec093f9-8c33-4d1a-b1a2-e8ae8357ddaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:13:47.964841 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec093f9-8c33-4d1a-b1a2-e8ae8357ddaa admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:13:47.965128 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec093f9-8c33-4d1a-b1a2-e8ae8357ddaa admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:13:47.965406 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec093f9-8c33-4d1a-b1a2-e8ae8357ddaa admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:13:47.966108 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec093f9-8c33-4d1a-b1a2-e8ae8357ddaa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:13:47.968034 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec093f9-8c33-4d1a-b1a2-e8ae8357ddaa admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:13:47.993452 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1181/2370] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 14:13:47 2022] POST /identity/v3/users => generated 307 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:13:47.998318 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e402e99f-7c56-48dd-912c-857ebcb8649c admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:13:48.009048 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e402e99f-7c56-48dd-912c-857ebcb8649c admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:13:48.014472 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18121f0-982d-4e38-b480-bc588f1e8c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:13:48.015141 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18121f0-982d-4e38-b480-bc588f1e8c58 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:13:48.015425 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18121f0-982d-4e38-b480-bc588f1e8c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:13:48.015700 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18121f0-982d-4e38-b480-bc588f1e8c58 admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:13:48.016403 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18121f0-982d-4e38-b480-bc588f1e8c58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:13:48.018108 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18121f0-982d-4e38-b480-bc588f1e8c58 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:13:48.029677 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1190/2371] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:13:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:13:48.034035 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec093f9-8c33-4d1a-b1a2-e8ae8357ddaa admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:13:48.047624 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec093f9-8c33-4d1a-b1a2-e8ae8357ddaa admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:13:48.054054 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f217b3ae-6517-4846-872a-b3b9f4d0055e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:13:48.054749 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f217b3ae-6517-4846-872a-b3b9f4d0055e admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:13:48.055037 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f217b3ae-6517-4846-872a-b3b9f4d0055e admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:13:48.055319 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f217b3ae-6517-4846-872a-b3b9f4d0055e admin admin] PATH_INFO: `/v3/projects/750d8a395bec432c9ecad661411d15e1/users/c3c53d5c80cd4488851425ab608726c7/roles/6e0201a9a6484f7d8b112b1eafff70bd` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:13:48.071790 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f217b3ae-6517-4846-872a-b3b9f4d0055e admin admin] RBAC: Authorizing `identity:create_grant(project_id=750d8a395bec432c9ecad661411d15e1, user_id=c3c53d5c80cd4488851425ab608726c7, role_id=6e0201a9a6484f7d8b112b1eafff70bd)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:13:48.074531 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f217b3ae-6517-4846-872a-b3b9f4d0055e admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:13:48.088198 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1182/2372] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 14:13:48 2022] PUT /identity/v3/projects/750d8a395bec432c9ecad661411d15e1/users/c3c53d5c80cd4488851425ab608726c7/roles/6e0201a9a6484f7d8b112b1eafff70bd => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:13:48.092361 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18121f0-982d-4e38-b480-bc588f1e8c58 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:13:48.102565 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18121f0-982d-4e38-b480-bc588f1e8c58 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:13:48.166158 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94bbcb2-3f01-469d-b688-3018d4bf3f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:13:48.166640 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94bbcb2-3f01-469d-b688-3018d4bf3f15 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:13:48.166926 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94bbcb2-3f01-469d-b688-3018d4bf3f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:13:48.167206 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94bbcb2-3f01-469d-b688-3018d4bf3f15 admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:13:48.167917 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94bbcb2-3f01-469d-b688-3018d4bf3f15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:13:48.169590 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94bbcb2-3f01-469d-b688-3018d4bf3f15 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:13:48.180190 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1191/2373] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:13:48 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:13:48.184496 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f217b3ae-6517-4846-872a-b3b9f4d0055e admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:13:48.194872 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f217b3ae-6517-4846-872a-b3b9f4d0055e admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:13:48.200196 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff60dfcf-93a1-4a2a-a182-ad03eb97fb66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:13:48.200879 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff60dfcf-93a1-4a2a-a182-ad03eb97fb66 admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:13:48.201163 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff60dfcf-93a1-4a2a-a182-ad03eb97fb66 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:13:48.201441 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff60dfcf-93a1-4a2a-a182-ad03eb97fb66 admin admin] PATH_INFO: `/v3/projects/750d8a395bec432c9ecad661411d15e1/users/c3c53d5c80cd4488851425ab608726c7/roles/6e0201a9a6484f7d8b112b1eafff70bd` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:13:48.203441 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff60dfcf-93a1-4a2a-a182-ad03eb97fb66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=750d8a395bec432c9ecad661411d15e1, user_id=c3c53d5c80cd4488851425ab608726c7, role_id=6e0201a9a6484f7d8b112b1eafff70bd)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:13:48.206126 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff60dfcf-93a1-4a2a-a182-ad03eb97fb66 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:13:48.220034 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1183/2374] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 14:13:48 2022] PUT /identity/v3/projects/750d8a395bec432c9ecad661411d15e1/users/c3c53d5c80cd4488851425ab608726c7/roles/6e0201a9a6484f7d8b112b1eafff70bd => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:13:48.230991 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89160ccc-c3e0-4457-8818-b24f89882e2d None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:13:48.231279 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89160ccc-c3e0-4457-8818-b24f89882e2d None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:13:48.231559 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89160ccc-c3e0-4457-8818-b24f89882e2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:13:48.262436 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-89160ccc-c3e0-4457-8818-b24f89882e2d None None] MFA Rules not processed for user `c3c53d5c80cd4488851425ab608726c7`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:13:48.263659 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-89160ccc-c3e0-4457-8818-b24f89882e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:13:48.337867 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1192/2375] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:13:48 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:13:58.216183 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=61,keystone:INSERT=5 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:13:58.334323 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=77,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:14:42.304912 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff60dfcf-93a1-4a2a-a182-ad03eb97fb66 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:14:42.306749 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-ff60dfcf-93a1-4a2a-a182-ad03eb97fb66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:14:42.371979 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff60dfcf-93a1-4a2a-a182-ad03eb97fb66 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:14:42.377757 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6a51ee-ca87-4554-8aad-9ccca826a8d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:14:42.378764 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6a51ee-ca87-4554-8aad-9ccca826a8d0 service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:14:42.379194 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6a51ee-ca87-4554-8aad-9ccca826a8d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:14:42.379468 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6a51ee-ca87-4554-8aad-9ccca826a8d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:14:42.381488 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-7f6a51ee-ca87-4554-8aad-9ccca826a8d0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:14:42.395440 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-7f6a51ee-ca87-4554-8aad-9ccca826a8d0 service cinder] Could not find project: 655f3d6aa32c49168d96fe9973204658.: keystone.exception.ProjectNotFound: Could not find project: 655f3d6aa32c49168d96fe9973204658. Jul 26 14:14:42.395440 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:14:42.395440 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:14:42.395440 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:14:42.395440 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:14:42.395440 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:14:42.395440 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:14:42.395440 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:14:42.395440 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:14:42.395440 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:14:42.395440 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:14:42.395440 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:14:42.395440 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Jul 26 14:14:42.395440 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Jul 26 14:14:42.395440 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Jul 26 14:14:42.396888 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Jul 26 14:14:42.396888 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Jul 26 14:14:42.396888 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Jul 26 14:14:42.396888 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:14:42.396888 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:14:42.396888 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token Jul 26 14:14:42.396888 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application token = self._validate_token(token_id) Jul 26 14:14:42.396888 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:14:42.396888 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:14:42.396888 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:14:42.396888 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:14:42.396888 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:14:42.396888 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 14:14:42.396888 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:14:42.396888 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:14:42.396888 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:14:42.396888 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:14:42.398289 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:14:42.398289 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:14:42.398289 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:14:42.398289 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:14:42.398289 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token Jul 26 14:14:42.398289 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) Jul 26 14:14:42.398289 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint Jul 26 14:14:42.398289 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application self._validate_token_resources() Jul 26 14:14:42.398289 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources Jul 26 14:14:42.398289 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): Jul 26 14:14:42.398289 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project Jul 26 14:14:42.398289 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( Jul 26 14:14:42.398289 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:14:42.398289 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:14:42.398289 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:14:42.398289 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:14:42.398289 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:14:42.399526 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:14:42.399526 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:14:42.399526 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 26 14:14:42.399526 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:14:42.399526 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:14:42.399526 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:14:42.399526 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:14:42.399526 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:14:42.399526 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:14:42.399526 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:14:42.399526 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:14:42.399526 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 14:14:42.399526 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 14:14:42.399526 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 14:14:42.399526 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 14:14:42.399526 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 14:14:42.399526 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 14:14:42.400670 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 655f3d6aa32c49168d96fe9973204658. Jul 26 14:14:42.400670 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 26 14:14:42.400670 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1184/2376] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:14:42 2022] GET /identity/v3/auth/tokens => generated 113 bytes in 92 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Jul 26 14:14:42.596151 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89160ccc-c3e0-4457-8818-b24f89882e2d None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:14:42.610265 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89160ccc-c3e0-4457-8818-b24f89882e2d None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:14:42.614700 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3258fe-99cf-47ba-9fe3-431ab6549980 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:14:42.615682 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3258fe-99cf-47ba-9fe3-431ab6549980 service cinder] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:14:42.616162 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3258fe-99cf-47ba-9fe3-431ab6549980 service cinder] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:14:42.616655 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3258fe-99cf-47ba-9fe3-431ab6549980 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:14:42.631358 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3258fe-99cf-47ba-9fe3-431ab6549980 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:14:42.633797 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3258fe-99cf-47ba-9fe3-431ab6549980 service cinder] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:14:42.648790 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1193/2377] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:14:42 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:14:52.386064 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=24 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:14:52.641853 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=6 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:15:49.287770 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6a51ee-ca87-4554-8aad-9ccca826a8d0 service cinder] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:15:49.289256 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-7f6a51ee-ca87-4554-8aad-9ccca826a8d0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:15:49.411494 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6a51ee-ca87-4554-8aad-9ccca826a8d0 service cinder] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:15:49.416698 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f41335b-bc8b-41ed-8496-e975e390103d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '835cbc13756b41b598142df3e35f5492', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:15:49.417325 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f41335b-bc8b-41ed-8496-e975e390103d service placement] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:15:49.417646 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f41335b-bc8b-41ed-8496-e975e390103d service placement] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:15:49.417926 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f41335b-bc8b-41ed-8496-e975e390103d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:15:49.419725 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-3f41335b-bc8b-41ed-8496-e975e390103d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:15:49.431576 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f41335b-bc8b-41ed-8496-e975e390103d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:15:49.433161 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f41335b-bc8b-41ed-8496-e975e390103d service placement] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:15:49.447023 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1185/2378] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:15:49 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:15:59.439400 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=52 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:16:38.033112 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3258fe-99cf-47ba-9fe3-431ab6549980 service cinder] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:16:38.034596 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-bf3258fe-99cf-47ba-9fe3-431ab6549980 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:16:38.090105 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3258fe-99cf-47ba-9fe3-431ab6549980 service cinder] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:16:38.093661 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b5aa80-e924-45db-8b53-38b8c2caca93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:16:38.094269 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b5aa80-e924-45db-8b53-38b8c2caca93 service neutron] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:16:38.094548 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b5aa80-e924-45db-8b53-38b8c2caca93 service neutron] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:16:38.094818 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b5aa80-e924-45db-8b53-38b8c2caca93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:16:38.096529 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-31b5aa80-e924-45db-8b53-38b8c2caca93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:16:38.186625 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b5aa80-e924-45db-8b53-38b8c2caca93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:16:38.188050 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b5aa80-e924-45db-8b53-38b8c2caca93 service neutron] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:16:38.208344 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1194/2379] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:16:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:16:42.722125 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f41335b-bc8b-41ed-8496-e975e390103d service placement] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:16:42.742823 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f41335b-bc8b-41ed-8496-e975e390103d service placement] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:16:42.746420 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22793717-e12a-4519-84dd-f57e563548cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:16:42.747035 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22793717-e12a-4519-84dd-f57e563548cd service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:16:42.747314 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22793717-e12a-4519-84dd-f57e563548cd service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:16:42.747588 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22793717-e12a-4519-84dd-f57e563548cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:16:42.806687 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22793717-e12a-4519-84dd-f57e563548cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:16:42.808115 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22793717-e12a-4519-84dd-f57e563548cd service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:16:42.816926 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1186/2380] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:16:42 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:16:43.054316 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b5aa80-e924-45db-8b53-38b8c2caca93 service neutron] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:16:43.057168 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-31b5aa80-e924-45db-8b53-38b8c2caca93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:16:43.095750 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b5aa80-e924-45db-8b53-38b8c2caca93 service neutron] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:16:43.118620 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a638cac5-6a03-4a3f-abb6-8c110caafe74 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c69161cd520b4b29bf50f3bf2f1c033b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:16:43.119277 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a638cac5-6a03-4a3f-abb6-8c110caafe74 service nova] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:16:43.120278 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a638cac5-6a03-4a3f-abb6-8c110caafe74 service nova] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:16:43.121550 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a638cac5-6a03-4a3f-abb6-8c110caafe74 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:16:43.131371 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-a638cac5-6a03-4a3f-abb6-8c110caafe74 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:16:43.163459 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a638cac5-6a03-4a3f-abb6-8c110caafe74 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:16:43.165360 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a638cac5-6a03-4a3f-abb6-8c110caafe74 service nova] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:16:43.186430 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22793717-e12a-4519-84dd-f57e563548cd service cinder] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:16:43.190157 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1195/2381] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:16:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:16:43.204532 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22793717-e12a-4519-84dd-f57e563548cd service cinder] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:16:43.211846 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09076ed-baec-4ed4-b576-07f18210a256 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:16:43.213461 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09076ed-baec-4ed4-b576-07f18210a256 service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:16:43.214611 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09076ed-baec-4ed4-b576-07f18210a256 service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:16:43.214772 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09076ed-baec-4ed4-b576-07f18210a256 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:16:43.217665 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-c09076ed-baec-4ed4-b576-07f18210a256 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:16:43.341195 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09076ed-baec-4ed4-b576-07f18210a256 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:16:43.343895 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09076ed-baec-4ed4-b576-07f18210a256 service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:16:43.360540 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1187/2382] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:16:43 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:16:43.417426 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a638cac5-6a03-4a3f-abb6-8c110caafe74 service nova] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:16:43.435833 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a638cac5-6a03-4a3f-abb6-8c110caafe74 service nova] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:16:43.443470 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a29df6-f6f6-4652-b248-2da1eb565b56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:16:43.444634 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a29df6-f6f6-4652-b248-2da1eb565b56 service cinder] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:16:43.444944 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a29df6-f6f6-4652-b248-2da1eb565b56 service cinder] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:16:43.445417 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a29df6-f6f6-4652-b248-2da1eb565b56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:16:43.449595 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-f1a29df6-f6f6-4652-b248-2da1eb565b56 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:16:43.514229 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:16:43.545961 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a29df6-f6f6-4652-b248-2da1eb565b56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:16:43.547640 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a29df6-f6f6-4652-b248-2da1eb565b56 service cinder] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:16:43.573606 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1196/2383] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:16:43 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:16:44.839062 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356d4d3a-85ee-4125-862b-8dc600deae3b None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:16:44.839622 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356d4d3a-85ee-4125-862b-8dc600deae3b None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:16:44.839749 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356d4d3a-85ee-4125-862b-8dc600deae3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:16:44.867830 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-356d4d3a-85ee-4125-862b-8dc600deae3b None None] MFA Rules not processed for user `bd9ee1f0f2354f048dd6e809121fa0aa`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:16:44.869074 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-356d4d3a-85ee-4125-862b-8dc600deae3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:16:44.926572 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1188/2384] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:16:44 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:16:45.283452 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a29df6-f6f6-4652-b248-2da1eb565b56 service cinder] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:16:45.305339 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a29df6-f6f6-4652-b248-2da1eb565b56 service cinder] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:16:45.309717 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127085a6-adde-49a3-94f2-48f48277635d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:16:45.310271 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127085a6-adde-49a3-94f2-48f48277635d service neutron] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:16:45.310555 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127085a6-adde-49a3-94f2-48f48277635d service neutron] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:16:45.310828 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127085a6-adde-49a3-94f2-48f48277635d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:16:45.312678 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-127085a6-adde-49a3-94f2-48f48277635d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:16:45.323377 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127085a6-adde-49a3-94f2-48f48277635d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:16:45.324817 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127085a6-adde-49a3-94f2-48f48277635d service neutron] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:16:45.334681 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1197/2385] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:16:45 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:16:45.675573 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356d4d3a-85ee-4125-862b-8dc600deae3b None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:16:45.677238 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-356d4d3a-85ee-4125-862b-8dc600deae3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:16:45.692377 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356d4d3a-85ee-4125-862b-8dc600deae3b None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:16:45.697594 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8549953a-ffa5-4229-94b4-cda1d1175e32 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '835cbc13756b41b598142df3e35f5492', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:16:45.698222 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8549953a-ffa5-4229-94b4-cda1d1175e32 service placement] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:16:45.698500 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8549953a-ffa5-4229-94b4-cda1d1175e32 service placement] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:16:45.698803 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8549953a-ffa5-4229-94b4-cda1d1175e32 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:16:45.700521 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-8549953a-ffa5-4229-94b4-cda1d1175e32 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:16:45.714146 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8549953a-ffa5-4229-94b4-cda1d1175e32 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:16:45.715729 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8549953a-ffa5-4229-94b4-cda1d1175e32 service placement] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:16:45.727090 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1189/2386] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:16:45 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:16:48.396252 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127085a6-adde-49a3-94f2-48f48277635d service neutron] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:16:48.412461 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127085a6-adde-49a3-94f2-48f48277635d service neutron] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:16:48.444658 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ad5e13-13d9-4561-b9d0-703ec2b36b55 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c69161cd520b4b29bf50f3bf2f1c033b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:16:48.445321 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ad5e13-13d9-4561-b9d0-703ec2b36b55 service nova] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:16:48.445684 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ad5e13-13d9-4561-b9d0-703ec2b36b55 service nova] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:16:48.445962 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ad5e13-13d9-4561-b9d0-703ec2b36b55 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:16:48.448053 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-19ad5e13-13d9-4561-b9d0-703ec2b36b55 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:16:48.466518 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ad5e13-13d9-4561-b9d0-703ec2b36b55 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:16:48.468150 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ad5e13-13d9-4561-b9d0-703ec2b36b55 service nova] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:16:48.501003 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1198/2387] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:16:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:16:53.649764 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8549953a-ffa5-4229-94b4-cda1d1175e32 service placement] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:16:53.663211 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8549953a-ffa5-4229-94b4-cda1d1175e32 service placement] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:16:53.668791 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97354e5-dd8e-464d-9da6-b15802db6afb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c69161cd520b4b29bf50f3bf2f1c033b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:16:53.669415 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97354e5-dd8e-464d-9da6-b15802db6afb service nova] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:16:53.669730 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97354e5-dd8e-464d-9da6-b15802db6afb service nova] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:16:53.670011 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97354e5-dd8e-464d-9da6-b15802db6afb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:16:53.672253 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-f97354e5-dd8e-464d-9da6-b15802db6afb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:16:53.687599 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97354e5-dd8e-464d-9da6-b15802db6afb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:16:53.689220 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97354e5-dd8e-464d-9da6-b15802db6afb service nova] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:16:53.695268 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:16:53.702884 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1190/2388] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:16:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:16:58.483862 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=27 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:17:03.703389 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:18:43.976641 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ad5e13-13d9-4561-b9d0-703ec2b36b55 service nova] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:18:44.003630 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ad5e13-13d9-4561-b9d0-703ec2b36b55 service nova] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:18:44.008512 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856ac6df-13de-4420-b4b0-d7f3effff967 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:18:44.009213 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856ac6df-13de-4420-b4b0-d7f3effff967 service cinder] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:18:44.009589 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856ac6df-13de-4420-b4b0-d7f3effff967 service cinder] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:18:44.009871 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856ac6df-13de-4420-b4b0-d7f3effff967 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:18:44.012573 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-856ac6df-13de-4420-b4b0-d7f3effff967 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:18:44.085364 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856ac6df-13de-4420-b4b0-d7f3effff967 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:18:44.087005 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856ac6df-13de-4420-b4b0-d7f3effff967 service cinder] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:18:44.097607 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1199/2389] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:18:43 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:18:44.277671 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97354e5-dd8e-464d-9da6-b15802db6afb service nova] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:18:44.292414 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97354e5-dd8e-464d-9da6-b15802db6afb service nova] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:18:44.296356 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8a7fed-3eb0-4df8-9655-e969f2a9a436 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:18:44.296985 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8a7fed-3eb0-4df8-9655-e969f2a9a436 service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:18:44.297294 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8a7fed-3eb0-4df8-9655-e969f2a9a436 service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:18:44.297612 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8a7fed-3eb0-4df8-9655-e969f2a9a436 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:18:44.299668 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-8f8a7fed-3eb0-4df8-9655-e969f2a9a436 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:18:44.323008 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8a7fed-3eb0-4df8-9655-e969f2a9a436 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:18:44.324492 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8a7fed-3eb0-4df8-9655-e969f2a9a436 service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:18:44.345268 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1191/2390] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:18:44 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 14:18:44.658739 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856ac6df-13de-4420-b4b0-d7f3effff967 service cinder] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:18:44.765290 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856ac6df-13de-4420-b4b0-d7f3effff967 service cinder] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:18:44.775430 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6b0223-56fa-4f2a-ac60-6486ffea7491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:18:44.776507 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6b0223-56fa-4f2a-ac60-6486ffea7491 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:18:44.777138 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6b0223-56fa-4f2a-ac60-6486ffea7491 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:18:44.777324 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6b0223-56fa-4f2a-ac60-6486ffea7491 admin admin] PATH_INFO: `/v3/users/86a2359178c04c70817c04d91eb6e986` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:18:44.778814 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6b0223-56fa-4f2a-ac60-6486ffea7491 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86a2359178c04c70817c04d91eb6e986)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:18:44.781090 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6b0223-56fa-4f2a-ac60-6486ffea7491 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:18:44.870486 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-0d6b0223-56fa-4f2a-ac60-6486ffea7491 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86a2359178c04c70817c04d91eb6e986'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:18:44.882621 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-0d6b0223-56fa-4f2a-ac60-6486ffea7491 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86a2359178c04c70817c04d91eb6e986'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:18:44.899614 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-0d6b0223-56fa-4f2a-ac60-6486ffea7491 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86a2359178c04c70817c04d91eb6e986'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:18:44.900686 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-0d6b0223-56fa-4f2a-ac60-6486ffea7491 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86a2359178c04c70817c04d91eb6e986'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:18:44.901416 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-0d6b0223-56fa-4f2a-ac60-6486ffea7491 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86a2359178c04c70817c04d91eb6e986'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:18:44.941565 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1200/2391] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 14:18:44 2022] DELETE /identity/v3/users/86a2359178c04c70817c04d91eb6e986 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:18:44.947154 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d703534-db33-4da4-b147-e0d1e73694f8 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:18:44.947154 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d703534-db33-4da4-b147-e0d1e73694f8 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:18:44.947154 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d703534-db33-4da4-b147-e0d1e73694f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:18:44.975019 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-2d703534-db33-4da4-b147-e0d1e73694f8 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:18:44.976810 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-2d703534-db33-4da4-b147-e0d1e73694f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:18:45.079114 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1192/2392] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:18:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:18:45.090803 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6b0223-56fa-4f2a-ac60-6486ffea7491 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:18:45.092643 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-0d6b0223-56fa-4f2a-ac60-6486ffea7491 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:18:45.127034 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:18:45.203338 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6b0223-56fa-4f2a-ac60-6486ffea7491 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:18:45.218011 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be46d3b6-525a-4ed2-80c5-8dfb73eb92db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:18:45.218670 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be46d3b6-525a-4ed2-80c5-8dfb73eb92db service neutron] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:18:45.219064 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be46d3b6-525a-4ed2-80c5-8dfb73eb92db service neutron] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:18:45.219741 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be46d3b6-525a-4ed2-80c5-8dfb73eb92db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:18:45.237870 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be46d3b6-525a-4ed2-80c5-8dfb73eb92db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:18:45.239857 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be46d3b6-525a-4ed2-80c5-8dfb73eb92db service neutron] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:18:45.253547 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1201/2393] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:18:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:18:45.682683 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d703534-db33-4da4-b147-e0d1e73694f8 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:18:45.684550 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-2d703534-db33-4da4-b147-e0d1e73694f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:18:45.704445 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d703534-db33-4da4-b147-e0d1e73694f8 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:18:45.714642 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a364ab-483f-415e-bfa9-631032945802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:18:45.715624 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a364ab-483f-415e-bfa9-631032945802 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:18:45.715930 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a364ab-483f-415e-bfa9-631032945802 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:18:45.716774 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a364ab-483f-415e-bfa9-631032945802 admin admin] PATH_INFO: `/v3/projects/48c5bfd0920f4ffe85dbdce45eb4723a` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:18:45.719340 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a364ab-483f-415e-bfa9-631032945802 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48c5bfd0920f4ffe85dbdce45eb4723a)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:18:45.724019 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a364ab-483f-415e-bfa9-631032945802 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:18:45.799836 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-e0a364ab-483f-415e-bfa9-631032945802 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48c5bfd0920f4ffe85dbdce45eb4723a', 'request_id': 'req-e0a364ab-483f-415e-bfa9-631032945802'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:18:45.810549 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-e0a364ab-483f-415e-bfa9-631032945802 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48c5bfd0920f4ffe85dbdce45eb4723a', 'request_id': 'req-e0a364ab-483f-415e-bfa9-631032945802'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:18:45.820739 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-e0a364ab-483f-415e-bfa9-631032945802 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48c5bfd0920f4ffe85dbdce45eb4723a', 'request_id': 'req-e0a364ab-483f-415e-bfa9-631032945802'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:18:45.845771 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-e0a364ab-483f-415e-bfa9-631032945802 admin admin] The token cache is being invalidate because project 48c5bfd0920f4ffe85dbdce45eb4723a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:18:45.846105 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-e0a364ab-483f-415e-bfa9-631032945802 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:18:45.846950 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-e0a364ab-483f-415e-bfa9-631032945802 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:18:45.848669 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1193/2394] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 14:18:45 2022] DELETE /identity/v3/projects/48c5bfd0920f4ffe85dbdce45eb4723a => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:18:45.852825 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be46d3b6-525a-4ed2-80c5-8dfb73eb92db service neutron] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:18:45.863768 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be46d3b6-525a-4ed2-80c5-8dfb73eb92db service neutron] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:18:45.940780 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d269f5-64e1-4a65-b3e4-cb665690bf67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:18:45.941504 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d269f5-64e1-4a65-b3e4-cb665690bf67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:18:45.941799 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d269f5-64e1-4a65-b3e4-cb665690bf67 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:18:45.942159 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d269f5-64e1-4a65-b3e4-cb665690bf67 admin admin] PATH_INFO: `/v3/users/c3c53d5c80cd4488851425ab608726c7` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:18:45.944015 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d269f5-64e1-4a65-b3e4-cb665690bf67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3c53d5c80cd4488851425ab608726c7)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:18:45.946857 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d269f5-64e1-4a65-b3e4-cb665690bf67 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:18:46.024601 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-c2d269f5-64e1-4a65-b3e4-cb665690bf67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3c53d5c80cd4488851425ab608726c7'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:18:46.036827 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-c2d269f5-64e1-4a65-b3e4-cb665690bf67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3c53d5c80cd4488851425ab608726c7'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:18:46.049863 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-c2d269f5-64e1-4a65-b3e4-cb665690bf67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3c53d5c80cd4488851425ab608726c7'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:18:46.050618 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-c2d269f5-64e1-4a65-b3e4-cb665690bf67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3c53d5c80cd4488851425ab608726c7'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:18:46.051383 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-c2d269f5-64e1-4a65-b3e4-cb665690bf67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3c53d5c80cd4488851425ab608726c7'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:18:46.076147 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1202/2395] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 14:18:45 2022] DELETE /identity/v3/users/c3c53d5c80cd4488851425ab608726c7 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:18:46.435429 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a364ab-483f-415e-bfa9-631032945802 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:18:46.439029 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-e0a364ab-483f-415e-bfa9-631032945802 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:18:46.466079 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a364ab-483f-415e-bfa9-631032945802 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:18:46.558447 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:18:46.582845 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a36ac24-e8f1-4c7f-8c59-908aef167482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:18:46.584118 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a36ac24-e8f1-4c7f-8c59-908aef167482 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:18:46.584755 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a36ac24-e8f1-4c7f-8c59-908aef167482 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:18:46.585692 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a36ac24-e8f1-4c7f-8c59-908aef167482 admin admin] PATH_INFO: `/v3/projects/750d8a395bec432c9ecad661411d15e1` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:18:46.588772 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a36ac24-e8f1-4c7f-8c59-908aef167482 admin admin] RBAC: Authorizing `identity:delete_project(project_id=750d8a395bec432c9ecad661411d15e1)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:18:46.593703 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a36ac24-e8f1-4c7f-8c59-908aef167482 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:18:46.703621 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-0a36ac24-e8f1-4c7f-8c59-908aef167482 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '750d8a395bec432c9ecad661411d15e1', 'request_id': 'req-0a36ac24-e8f1-4c7f-8c59-908aef167482'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:18:46.716397 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-0a36ac24-e8f1-4c7f-8c59-908aef167482 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '750d8a395bec432c9ecad661411d15e1', 'request_id': 'req-0a36ac24-e8f1-4c7f-8c59-908aef167482'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:18:46.725788 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-0a36ac24-e8f1-4c7f-8c59-908aef167482 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '750d8a395bec432c9ecad661411d15e1', 'request_id': 'req-0a36ac24-e8f1-4c7f-8c59-908aef167482'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:18:46.751615 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-0a36ac24-e8f1-4c7f-8c59-908aef167482 admin admin] The token cache is being invalidate because project 750d8a395bec432c9ecad661411d15e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:18:46.752584 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-0a36ac24-e8f1-4c7f-8c59-908aef167482 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:18:46.754557 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-0a36ac24-e8f1-4c7f-8c59-908aef167482 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:18:46.763059 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1194/2396] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 14:18:46 2022] DELETE /identity/v3/projects/750d8a395bec432c9ecad661411d15e1 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:18:46.765084 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cecfa92-af0f-408a-a9dc-3647181c7811 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:18:46.766058 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cecfa92-af0f-408a-a9dc-3647181c7811 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:18:46.766058 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cecfa92-af0f-408a-a9dc-3647181c7811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:18:46.808118 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-9cecfa92-af0f-408a-a9dc-3647181c7811 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:18:46.810238 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-9cecfa92-af0f-408a-a9dc-3647181c7811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:18:46.849600 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:18:46.902519 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1203/2397] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:18:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:18:46.910399 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99441b9c-4102-47eb-9c63-d4ee99a457f2 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:18:46.911436 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99441b9c-4102-47eb-9c63-d4ee99a457f2 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:18:46.911891 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99441b9c-4102-47eb-9c63-d4ee99a457f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:18:46.942019 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-99441b9c-4102-47eb-9c63-d4ee99a457f2 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:18:46.945574 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-99441b9c-4102-47eb-9c63-d4ee99a457f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:18:46.953092 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1195/2398] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:18:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:18:46.958902 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cecfa92-af0f-408a-a9dc-3647181c7811 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:18:46.972758 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cecfa92-af0f-408a-a9dc-3647181c7811 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:18:46.977906 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e395f1-c789-42bc-b60a-022c2c3ed203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:18:46.979281 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e395f1-c789-42bc-b60a-022c2c3ed203 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:18:46.979670 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e395f1-c789-42bc-b60a-022c2c3ed203 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:18:46.980003 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e395f1-c789-42bc-b60a-022c2c3ed203 admin admin] PATH_INFO: `/v3/domains` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:18:46.980789 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e395f1-c789-42bc-b60a-022c2c3ed203 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:18:46.981104 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e395f1-c789-42bc-b60a-022c2c3ed203 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:18:46.982841 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e395f1-c789-42bc-b60a-022c2c3ed203 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:18:47.009607 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1204/2399] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Jul 26 14:18:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:18:47.014068 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99441b9c-4102-47eb-9c63-d4ee99a457f2 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:18:47.024927 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99441b9c-4102-47eb-9c63-d4ee99a457f2 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:18:47.030678 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1877fbd-9081-41c9-8a83-6149bb623515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:18:47.035681 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1877fbd-9081-41c9-8a83-6149bb623515 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:18:47.036152 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1877fbd-9081-41c9-8a83-6149bb623515 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:18:47.036719 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1877fbd-9081-41c9-8a83-6149bb623515 admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:18:47.037785 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1877fbd-9081-41c9-8a83-6149bb623515 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:18:47.039822 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1877fbd-9081-41c9-8a83-6149bb623515 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:18:47.080689 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1196/2400] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 14:18:47 2022] POST /identity/v3/projects => generated 379 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:18:47.087296 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e395f1-c789-42bc-b60a-022c2c3ed203 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:18:47.106680 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e395f1-c789-42bc-b60a-022c2c3ed203 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:18:47.190262 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea64694b-3a34-4c7b-afe7-8cee3d30df83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:18:47.190911 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea64694b-3a34-4c7b-afe7-8cee3d30df83 admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:18:47.191196 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea64694b-3a34-4c7b-afe7-8cee3d30df83 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:18:47.191508 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea64694b-3a34-4c7b-afe7-8cee3d30df83 admin admin] PATH_INFO: `/v3/users` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:18:47.192381 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea64694b-3a34-4c7b-afe7-8cee3d30df83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:18:47.195367 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea64694b-3a34-4c7b-afe7-8cee3d30df83 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:18:47.221376 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1205/2401] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 14:18:47 2022] POST /identity/v3/users => generated 290 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:18:47.226019 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1877fbd-9081-41c9-8a83-6149bb623515 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:18:47.237363 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1877fbd-9081-41c9-8a83-6149bb623515 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:18:47.244318 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fdbd1f-cff8-4c57-9880-71302b5602d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:18:47.244947 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fdbd1f-cff8-4c57-9880-71302b5602d9 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:18:47.245226 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fdbd1f-cff8-4c57-9880-71302b5602d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:18:47.245518 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fdbd1f-cff8-4c57-9880-71302b5602d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:18:47.246225 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fdbd1f-cff8-4c57-9880-71302b5602d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:18:47.248018 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fdbd1f-cff8-4c57-9880-71302b5602d9 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:18:47.259283 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1197/2402] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:18:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 14:18:47.263955 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea64694b-3a34-4c7b-afe7-8cee3d30df83 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:18:47.278202 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea64694b-3a34-4c7b-afe7-8cee3d30df83 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:18:47.284122 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cd5669-ac61-41fe-8719-6a9218500351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:18:47.284981 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cd5669-ac61-41fe-8719-6a9218500351 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:18:47.285284 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cd5669-ac61-41fe-8719-6a9218500351 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:18:47.285600 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cd5669-ac61-41fe-8719-6a9218500351 admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:18:47.286320 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cd5669-ac61-41fe-8719-6a9218500351 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:18:47.287945 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cd5669-ac61-41fe-8719-6a9218500351 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:18:47.302549 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1206/2403] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:18:47 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 14:18:47.307945 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fdbd1f-cff8-4c57-9880-71302b5602d9 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:18:47.320133 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fdbd1f-cff8-4c57-9880-71302b5602d9 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:18:47.329131 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6918b0a3-87fb-4220-8607-e1806fb303a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:18:47.330065 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6918b0a3-87fb-4220-8607-e1806fb303a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:18:47.330485 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6918b0a3-87fb-4220-8607-e1806fb303a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:18:47.331034 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6918b0a3-87fb-4220-8607-e1806fb303a0 admin admin] PATH_INFO: `/v3/projects/b077ab7ec08249e3bd91e6a37b131a79/users/952fb39bea8c40e29e0a4a7f0e0d91b0/roles/759f389b221049c4b01faf3b984cd8d7` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:18:47.348248 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6918b0a3-87fb-4220-8607-e1806fb303a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b077ab7ec08249e3bd91e6a37b131a79, user_id=952fb39bea8c40e29e0a4a7f0e0d91b0, role_id=759f389b221049c4b01faf3b984cd8d7)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:18:47.351251 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6918b0a3-87fb-4220-8607-e1806fb303a0 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:18:47.368978 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1198/2404] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 14:18:47 2022] PUT /identity/v3/projects/b077ab7ec08249e3bd91e6a37b131a79/users/952fb39bea8c40e29e0a4a7f0e0d91b0/roles/759f389b221049c4b01faf3b984cd8d7 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:18:47.382334 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e201a32d-92a9-4c81-bd0a-4daf2878ac16 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:18:47.382763 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e201a32d-92a9-4c81-bd0a-4daf2878ac16 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:18:47.383118 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e201a32d-92a9-4c81-bd0a-4daf2878ac16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:18:47.419694 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-e201a32d-92a9-4c81-bd0a-4daf2878ac16 None None] MFA Rules not processed for user `952fb39bea8c40e29e0a4a7f0e0d91b0`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:18:47.421198 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-e201a32d-92a9-4c81-bd0a-4daf2878ac16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:18:47.544458 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1207/2405] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:18:47 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:18:47.550075 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6918b0a3-87fb-4220-8607-e1806fb303a0 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:18:47.682725 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6918b0a3-87fb-4220-8607-e1806fb303a0 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:18:47.693976 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc17f84a-a208-47f3-be41-f064d9889eee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:18:47.694680 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc17f84a-a208-47f3-be41-f064d9889eee admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:18:47.694982 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc17f84a-a208-47f3-be41-f064d9889eee admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:18:47.695254 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc17f84a-a208-47f3-be41-f064d9889eee admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:18:47.696072 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc17f84a-a208-47f3-be41-f064d9889eee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:18:47.698611 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc17f84a-a208-47f3-be41-f064d9889eee admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:18:47.737131 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1199/2406] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 14:18:47 2022] POST /identity/v3/projects => generated 377 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:18:47.741674 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e201a32d-92a9-4c81-bd0a-4daf2878ac16 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:18:47.755043 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e201a32d-92a9-4c81-bd0a-4daf2878ac16 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:18:47.794192 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:18:47.854300 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c285f26e-c1e8-4509-9847-89ffc0f1603a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:18:47.855476 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c285f26e-c1e8-4509-9847-89ffc0f1603a admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:18:47.855793 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c285f26e-c1e8-4509-9847-89ffc0f1603a admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:18:47.856128 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c285f26e-c1e8-4509-9847-89ffc0f1603a admin admin] PATH_INFO: `/v3/users` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:18:47.856878 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c285f26e-c1e8-4509-9847-89ffc0f1603a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:18:47.859311 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c285f26e-c1e8-4509-9847-89ffc0f1603a admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:18:47.909089 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1208/2407] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 14:18:47 2022] POST /identity/v3/users => generated 289 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:18:47.912140 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc17f84a-a208-47f3-be41-f064d9889eee admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:18:47.930898 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:18:47.940752 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc17f84a-a208-47f3-be41-f064d9889eee admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:18:47.958833 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd4c9a4-d05b-4b1f-b9fe-ee26372d0f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:18:47.958833 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd4c9a4-d05b-4b1f-b9fe-ee26372d0f92 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:18:47.958833 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd4c9a4-d05b-4b1f-b9fe-ee26372d0f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:18:47.958833 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd4c9a4-d05b-4b1f-b9fe-ee26372d0f92 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:18:47.959667 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd4c9a4-d05b-4b1f-b9fe-ee26372d0f92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:18:47.959667 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd4c9a4-d05b-4b1f-b9fe-ee26372d0f92 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:18:47.977109 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1200/2408] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:18:47 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:18:47.979063 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c285f26e-c1e8-4509-9847-89ffc0f1603a admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:18:47.996130 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c285f26e-c1e8-4509-9847-89ffc0f1603a admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:18:48.011334 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b32828e-d5fe-4410-8f09-ad6172ed5cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:18:48.011334 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b32828e-d5fe-4410-8f09-ad6172ed5cc0 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:18:48.011813 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b32828e-d5fe-4410-8f09-ad6172ed5cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:18:48.011972 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b32828e-d5fe-4410-8f09-ad6172ed5cc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:18:48.012652 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b32828e-d5fe-4410-8f09-ad6172ed5cc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:18:48.014307 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b32828e-d5fe-4410-8f09-ad6172ed5cc0 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:18:48.032758 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1209/2409] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:18:47 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:18:48.035938 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd4c9a4-d05b-4b1f-b9fe-ee26372d0f92 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:18:48.062216 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd4c9a4-d05b-4b1f-b9fe-ee26372d0f92 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:18:48.070366 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65a68bf-4723-418d-8604-438d53e14212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:18:48.071708 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65a68bf-4723-418d-8604-438d53e14212 admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:18:48.072919 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65a68bf-4723-418d-8604-438d53e14212 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:18:48.073563 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65a68bf-4723-418d-8604-438d53e14212 admin admin] PATH_INFO: `/v3/projects/75718c4f49434ff3853eddaa1209bf1b/users/941f9346e3a94d818c5aa3bd184f6976/roles/759f389b221049c4b01faf3b984cd8d7` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:18:48.108797 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65a68bf-4723-418d-8604-438d53e14212 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75718c4f49434ff3853eddaa1209bf1b, user_id=941f9346e3a94d818c5aa3bd184f6976, role_id=759f389b221049c4b01faf3b984cd8d7)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:18:48.114831 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65a68bf-4723-418d-8604-438d53e14212 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:18:48.137244 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1201/2410] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 14:18:48 2022] PUT /identity/v3/projects/75718c4f49434ff3853eddaa1209bf1b/users/941f9346e3a94d818c5aa3bd184f6976/roles/759f389b221049c4b01faf3b984cd8d7 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:18:48.150011 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a08148d-c657-4bee-9ca6-826befbf5368 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:18:48.151102 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a08148d-c657-4bee-9ca6-826befbf5368 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:18:48.151618 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a08148d-c657-4bee-9ca6-826befbf5368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:18:48.193925 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-6a08148d-c657-4bee-9ca6-826befbf5368 None None] MFA Rules not processed for user `941f9346e3a94d818c5aa3bd184f6976`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:18:48.195377 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-6a08148d-c657-4bee-9ca6-826befbf5368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:18:48.287342 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1210/2411] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:18:48 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:18:48.288671 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65a68bf-4723-418d-8604-438d53e14212 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:18:48.368126 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65a68bf-4723-418d-8604-438d53e14212 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:18:48.376671 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982d6241-e434-4d57-ae9d-1b02c5968085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:18:48.377652 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982d6241-e434-4d57-ae9d-1b02c5968085 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:18:48.378220 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982d6241-e434-4d57-ae9d-1b02c5968085 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:18:48.378539 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982d6241-e434-4d57-ae9d-1b02c5968085 admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:18:48.379149 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982d6241-e434-4d57-ae9d-1b02c5968085 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:18:48.381087 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982d6241-e434-4d57-ae9d-1b02c5968085 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:18:48.424990 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1202/2412] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 14:18:48 2022] POST /identity/v3/projects => generated 377 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:18:48.426969 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a08148d-c657-4bee-9ca6-826befbf5368 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:18:48.439501 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a08148d-c657-4bee-9ca6-826befbf5368 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:18:48.536768 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bc7f2c-a200-4d05-bc62-5eecee6f392d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:18:48.537742 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bc7f2c-a200-4d05-bc62-5eecee6f392d admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:18:48.538244 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bc7f2c-a200-4d05-bc62-5eecee6f392d admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:18:48.538738 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bc7f2c-a200-4d05-bc62-5eecee6f392d admin admin] PATH_INFO: `/v3/users` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:18:48.539989 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bc7f2c-a200-4d05-bc62-5eecee6f392d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:18:48.543851 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bc7f2c-a200-4d05-bc62-5eecee6f392d admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:18:48.576714 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1211/2413] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 14:18:48 2022] POST /identity/v3/users => generated 295 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:18:48.581677 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982d6241-e434-4d57-ae9d-1b02c5968085 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:18:48.596241 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982d6241-e434-4d57-ae9d-1b02c5968085 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:18:48.601180 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cef38a-4336-47aa-8b69-81de7438dd54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:18:48.601841 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cef38a-4336-47aa-8b69-81de7438dd54 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:18:48.602154 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cef38a-4336-47aa-8b69-81de7438dd54 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:18:48.602427 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cef38a-4336-47aa-8b69-81de7438dd54 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:18:48.603161 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cef38a-4336-47aa-8b69-81de7438dd54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:18:48.604791 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cef38a-4336-47aa-8b69-81de7438dd54 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:18:48.616086 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1203/2414] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:18:48 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 14:18:48.622043 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bc7f2c-a200-4d05-bc62-5eecee6f392d admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:18:48.636467 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bc7f2c-a200-4d05-bc62-5eecee6f392d admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:18:48.642026 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fa4195-db94-47b3-9381-4f6bead459fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:18:48.642726 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fa4195-db94-47b3-9381-4f6bead459fc admin admin] REQUEST_METHOD: `PUT` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:18:48.643692 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fa4195-db94-47b3-9381-4f6bead459fc admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:18:48.643692 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fa4195-db94-47b3-9381-4f6bead459fc admin admin] PATH_INFO: `/v3/projects/39c93df4f3a64417bb5337e3eae00cf7/users/c4e222a678ff4a44b74d252aa6621608/roles/6e0201a9a6484f7d8b112b1eafff70bd` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:18:48.662649 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fa4195-db94-47b3-9381-4f6bead459fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=39c93df4f3a64417bb5337e3eae00cf7, user_id=c4e222a678ff4a44b74d252aa6621608, role_id=6e0201a9a6484f7d8b112b1eafff70bd)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:18:48.665325 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fa4195-db94-47b3-9381-4f6bead459fc admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:18:48.675303 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:18:48.682469 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1212/2415] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 14:18:48 2022] PUT /identity/v3/projects/39c93df4f3a64417bb5337e3eae00cf7/users/c4e222a678ff4a44b74d252aa6621608/roles/6e0201a9a6484f7d8b112b1eafff70bd => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:18:48.686631 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cef38a-4336-47aa-8b69-81de7438dd54 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:18:48.699562 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cef38a-4336-47aa-8b69-81de7438dd54 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:18:48.777557 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eda64e1-6b0e-467b-a914-bdfe85f1670a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:18:48.778604 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eda64e1-6b0e-467b-a914-bdfe85f1670a admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:18:48.778604 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eda64e1-6b0e-467b-a914-bdfe85f1670a admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:18:48.778831 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eda64e1-6b0e-467b-a914-bdfe85f1670a admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:18:48.779792 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eda64e1-6b0e-467b-a914-bdfe85f1670a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:18:48.781514 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eda64e1-6b0e-467b-a914-bdfe85f1670a admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:18:48.793433 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1204/2416] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:18:48 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:18:48.798871 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fa4195-db94-47b3-9381-4f6bead459fc admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:18:48.814019 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fa4195-db94-47b3-9381-4f6bead459fc admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:18:48.819214 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1016ba2-b5f5-4ae7-afa1-653aa49e1fad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:18:48.819938 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1016ba2-b5f5-4ae7-afa1-653aa49e1fad admin admin] REQUEST_METHOD: `PUT` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:18:48.820214 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1016ba2-b5f5-4ae7-afa1-653aa49e1fad admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:18:48.820487 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1016ba2-b5f5-4ae7-afa1-653aa49e1fad admin admin] PATH_INFO: `/v3/projects/39c93df4f3a64417bb5337e3eae00cf7/users/c4e222a678ff4a44b74d252aa6621608/roles/6e0201a9a6484f7d8b112b1eafff70bd` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:18:48.823167 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1016ba2-b5f5-4ae7-afa1-653aa49e1fad admin admin] RBAC: Authorizing `identity:create_grant(project_id=39c93df4f3a64417bb5337e3eae00cf7, user_id=c4e222a678ff4a44b74d252aa6621608, role_id=6e0201a9a6484f7d8b112b1eafff70bd)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:18:48.826047 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1016ba2-b5f5-4ae7-afa1-653aa49e1fad admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:18:48.840395 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1213/2417] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 14:18:48 2022] PUT /identity/v3/projects/39c93df4f3a64417bb5337e3eae00cf7/users/c4e222a678ff4a44b74d252aa6621608/roles/6e0201a9a6484f7d8b112b1eafff70bd => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:18:48.856830 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac520d7-ab09-4925-871b-f5d309e54680 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:18:48.857506 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac520d7-ab09-4925-871b-f5d309e54680 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:18:48.858001 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac520d7-ab09-4925-871b-f5d309e54680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:18:48.907765 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-1ac520d7-ab09-4925-871b-f5d309e54680 None None] MFA Rules not processed for user `c4e222a678ff4a44b74d252aa6621608`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:18:48.909317 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-1ac520d7-ab09-4925-871b-f5d309e54680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:18:48.971654 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:18:49.059935 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1205/2418] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:18:48 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:18:58.834653 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=5,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:18:59.053943 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=16 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:20:36.189891 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1016ba2-b5f5-4ae7-afa1-653aa49e1fad admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:20:36.191778 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-a1016ba2-b5f5-4ae7-afa1-653aa49e1fad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:20:36.307737 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1016ba2-b5f5-4ae7-afa1-653aa49e1fad admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:20:36.312756 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145b97d9-24fe-482b-a8e7-8174f4d93645 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:20:36.313828 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b97d9-24fe-482b-a8e7-8174f4d93645 service cinder] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:36.314521 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b97d9-24fe-482b-a8e7-8174f4d93645 service cinder] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:36.315098 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b97d9-24fe-482b-a8e7-8174f4d93645 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:36.334351 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145b97d9-24fe-482b-a8e7-8174f4d93645 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:20:36.336213 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145b97d9-24fe-482b-a8e7-8174f4d93645 service cinder] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:20:36.337561 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac520d7-ab09-4925-871b-f5d309e54680 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:20:36.339680 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-1ac520d7-ab09-4925-871b-f5d309e54680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:20:36.353691 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1214/2419] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:20:36 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:20:36.360518 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac520d7-ab09-4925-871b-f5d309e54680 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:20:36.365084 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3f0075-83b8-400e-9383-0551f07c91d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:20:36.365813 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3f0075-83b8-400e-9383-0551f07c91d2 service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:36.366099 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3f0075-83b8-400e-9383-0551f07c91d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:36.366383 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3f0075-83b8-400e-9383-0551f07c91d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:36.377299 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3f0075-83b8-400e-9383-0551f07c91d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:20:36.379016 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3f0075-83b8-400e-9383-0551f07c91d2 service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:20:36.391674 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1206/2420] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:20:36 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 14:20:36.797183 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145b97d9-24fe-482b-a8e7-8174f4d93645 service cinder] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:20:36.916810 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145b97d9-24fe-482b-a8e7-8174f4d93645 service cinder] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:20:36.924047 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f509d7-296f-40cd-8594-bfddad09a247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:20:36.927316 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f509d7-296f-40cd-8594-bfddad09a247 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:36.927316 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f509d7-296f-40cd-8594-bfddad09a247 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:36.927316 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f509d7-296f-40cd-8594-bfddad09a247 admin admin] PATH_INFO: `/v3/users/952fb39bea8c40e29e0a4a7f0e0d91b0` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:36.927316 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f509d7-296f-40cd-8594-bfddad09a247 admin admin] RBAC: Authorizing `identity:delete_user(user_id=952fb39bea8c40e29e0a4a7f0e0d91b0)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:20:36.928898 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f509d7-296f-40cd-8594-bfddad09a247 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:20:36.992366 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-10f509d7-296f-40cd-8594-bfddad09a247 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '952fb39bea8c40e29e0a4a7f0e0d91b0'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:20:37.003051 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-10f509d7-296f-40cd-8594-bfddad09a247 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '952fb39bea8c40e29e0a4a7f0e0d91b0'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:20:37.011030 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-10f509d7-296f-40cd-8594-bfddad09a247 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '952fb39bea8c40e29e0a4a7f0e0d91b0'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:20:37.011743 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-10f509d7-296f-40cd-8594-bfddad09a247 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '952fb39bea8c40e29e0a4a7f0e0d91b0'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:20:37.012348 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-10f509d7-296f-40cd-8594-bfddad09a247 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '952fb39bea8c40e29e0a4a7f0e0d91b0'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:20:37.025512 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:20:37.034896 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1215/2421] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 14:20:36 2022] DELETE /identity/v3/users/952fb39bea8c40e29e0a4a7f0e0d91b0 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:20:37.041265 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f01aa2-aa0d-4b3c-be1a-f40f451a4ab4 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:37.041768 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f01aa2-aa0d-4b3c-be1a-f40f451a4ab4 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:37.042253 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f01aa2-aa0d-4b3c-be1a-f40f451a4ab4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:37.091083 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-83f01aa2-aa0d-4b3c-be1a-f40f451a4ab4 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:20:37.092824 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-83f01aa2-aa0d-4b3c-be1a-f40f451a4ab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:20:37.181415 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1207/2422] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:20:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:20:37.192824 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f509d7-296f-40cd-8594-bfddad09a247 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:20:37.194608 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-10f509d7-296f-40cd-8594-bfddad09a247 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:20:37.267113 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f509d7-296f-40cd-8594-bfddad09a247 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:20:37.271577 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a49a2d-d698-4055-b286-822d2a035d4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:20:37.272786 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a49a2d-d698-4055-b286-822d2a035d4a service neutron] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:37.272786 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a49a2d-d698-4055-b286-822d2a035d4a service neutron] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:37.273026 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a49a2d-d698-4055-b286-822d2a035d4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:37.284953 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a49a2d-d698-4055-b286-822d2a035d4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:20:37.286411 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a49a2d-d698-4055-b286-822d2a035d4a service neutron] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:20:37.298529 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1216/2423] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:20:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:20:37.707562 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f01aa2-aa0d-4b3c-be1a-f40f451a4ab4 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:20:37.709033 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-83f01aa2-aa0d-4b3c-be1a-f40f451a4ab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:20:37.736394 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f01aa2-aa0d-4b3c-be1a-f40f451a4ab4 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:20:37.742196 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5c80bd-4529-444e-bca5-39f5181d5251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:20:37.742918 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5c80bd-4529-444e-bca5-39f5181d5251 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:37.743201 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5c80bd-4529-444e-bca5-39f5181d5251 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:37.743476 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5c80bd-4529-444e-bca5-39f5181d5251 admin admin] PATH_INFO: `/v3/projects/b077ab7ec08249e3bd91e6a37b131a79` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:37.745143 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5c80bd-4529-444e-bca5-39f5181d5251 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b077ab7ec08249e3bd91e6a37b131a79)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:20:37.747139 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5c80bd-4529-444e-bca5-39f5181d5251 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:20:37.844744 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-fe5c80bd-4529-444e-bca5-39f5181d5251 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b077ab7ec08249e3bd91e6a37b131a79', 'request_id': 'req-fe5c80bd-4529-444e-bca5-39f5181d5251'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:20:37.859397 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-fe5c80bd-4529-444e-bca5-39f5181d5251 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b077ab7ec08249e3bd91e6a37b131a79', 'request_id': 'req-fe5c80bd-4529-444e-bca5-39f5181d5251'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:20:37.870265 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-fe5c80bd-4529-444e-bca5-39f5181d5251 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b077ab7ec08249e3bd91e6a37b131a79', 'request_id': 'req-fe5c80bd-4529-444e-bca5-39f5181d5251'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:20:37.896051 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-fe5c80bd-4529-444e-bca5-39f5181d5251 admin admin] The token cache is being invalidate because project b077ab7ec08249e3bd91e6a37b131a79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:20:37.896538 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-fe5c80bd-4529-444e-bca5-39f5181d5251 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:20:37.897223 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-fe5c80bd-4529-444e-bca5-39f5181d5251 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:20:37.898890 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1208/2424] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 14:20:37 2022] DELETE /identity/v3/projects/b077ab7ec08249e3bd91e6a37b131a79 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:20:37.902962 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a49a2d-d698-4055-b286-822d2a035d4a service neutron] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:20:37.906020 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5c80bd-4529-444e-bca5-39f5181d5251 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:20:37.907714 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-fe5c80bd-4529-444e-bca5-39f5181d5251 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:20:37.916980 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a49a2d-d698-4055-b286-822d2a035d4a service neutron] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:20:38.003366 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5c80bd-4529-444e-bca5-39f5181d5251 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:20:38.009288 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b18f26-7e93-4c58-9712-7091e59269cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:20:38.010343 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b18f26-7e93-4c58-9712-7091e59269cc service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:38.011051 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b18f26-7e93-4c58-9712-7091e59269cc service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:38.012160 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b18f26-7e93-4c58-9712-7091e59269cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:38.015650 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-33b18f26-7e93-4c58-9712-7091e59269cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:20:38.016514 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e7f8ed-1130-458b-9cdb-215704d23a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:20:38.017251 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e7f8ed-1130-458b-9cdb-215704d23a05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:38.017714 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e7f8ed-1130-458b-9cdb-215704d23a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:38.018092 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e7f8ed-1130-458b-9cdb-215704d23a05 admin admin] PATH_INFO: `/v3/users/941f9346e3a94d818c5aa3bd184f6976` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:38.019535 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e7f8ed-1130-458b-9cdb-215704d23a05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=941f9346e3a94d818c5aa3bd184f6976)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:20:38.021892 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e7f8ed-1130-458b-9cdb-215704d23a05 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:20:38.046565 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:20:38.052508 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b18f26-7e93-4c58-9712-7091e59269cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:20:38.054157 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b18f26-7e93-4c58-9712-7091e59269cc service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:20:38.069079 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1209/2425] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:20:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:20:38.115149 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b18f26-7e93-4c58-9712-7091e59269cc service neutron] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:20:38.119483 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-a2e7f8ed-1130-458b-9cdb-215704d23a05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '941f9346e3a94d818c5aa3bd184f6976'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:20:38.131372 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b18f26-7e93-4c58-9712-7091e59269cc service neutron] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:20:38.139421 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1a6ffd-4238-4aae-bb1a-cd077d7bae93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:20:38.140836 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1a6ffd-4238-4aae-bb1a-cd077d7bae93 service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:38.141684 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1a6ffd-4238-4aae-bb1a-cd077d7bae93 service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:38.142485 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1a6ffd-4238-4aae-bb1a-cd077d7bae93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:38.149161 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-0b1a6ffd-4238-4aae-bb1a-cd077d7bae93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:20:38.158835 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-a2e7f8ed-1130-458b-9cdb-215704d23a05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '941f9346e3a94d818c5aa3bd184f6976'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:20:38.174232 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1a6ffd-4238-4aae-bb1a-cd077d7bae93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:20:38.178091 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1a6ffd-4238-4aae-bb1a-cd077d7bae93 service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:20:38.181698 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-a2e7f8ed-1130-458b-9cdb-215704d23a05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '941f9346e3a94d818c5aa3bd184f6976'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:20:38.182822 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-a2e7f8ed-1130-458b-9cdb-215704d23a05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '941f9346e3a94d818c5aa3bd184f6976'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:20:38.183982 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-a2e7f8ed-1130-458b-9cdb-215704d23a05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '941f9346e3a94d818c5aa3bd184f6976'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:20:38.200509 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1210/2426] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:20:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:20:38.219303 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1217/2427] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 14:20:37 2022] DELETE /identity/v3/users/941f9346e3a94d818c5aa3bd184f6976 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:20:38.466069 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e02e172-66a2-4acf-8a5b-53d5967e864e None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:38.467293 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e02e172-66a2-4acf-8a5b-53d5967e864e None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:38.467550 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e02e172-66a2-4acf-8a5b-53d5967e864e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:38.583562 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-7e02e172-66a2-4acf-8a5b-53d5967e864e None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:20:38.585860 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-7e02e172-66a2-4acf-8a5b-53d5967e864e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:20:38.680159 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a84a561-1526-4c6a-bfcc-238c4e188952 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:38.680482 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a84a561-1526-4c6a-bfcc-238c4e188952 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:38.680767 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a84a561-1526-4c6a-bfcc-238c4e188952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:38.703949 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1211/2428] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:20:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:20:38.726255 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-0a84a561-1526-4c6a-bfcc-238c4e188952 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:20:38.727655 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-0a84a561-1526-4c6a-bfcc-238c4e188952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:20:38.731099 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e02e172-66a2-4acf-8a5b-53d5967e864e None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:20:38.732728 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-7e02e172-66a2-4acf-8a5b-53d5967e864e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:20:38.756999 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e02e172-66a2-4acf-8a5b-53d5967e864e None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:20:38.779559 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1218/2429] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:20:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:20:38.784889 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a84a561-1526-4c6a-bfcc-238c4e188952 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:20:38.787126 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60d0856-849b-477a-88f2-29b7b927b789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:20:38.788141 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60d0856-849b-477a-88f2-29b7b927b789 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:38.788643 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60d0856-849b-477a-88f2-29b7b927b789 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:38.789134 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60d0856-849b-477a-88f2-29b7b927b789 admin admin] PATH_INFO: `/v3/projects/75718c4f49434ff3853eddaa1209bf1b` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:38.790669 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60d0856-849b-477a-88f2-29b7b927b789 admin admin] RBAC: Authorizing `identity:delete_project(project_id=75718c4f49434ff3853eddaa1209bf1b)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:20:38.799894 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a84a561-1526-4c6a-bfcc-238c4e188952 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:20:38.803080 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60d0856-849b-477a-88f2-29b7b927b789 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:20:38.807656 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f438b3-7fe5-4987-badf-aff116693ff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:20:38.807926 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f438b3-7fe5-4987-badf-aff116693ff3 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:38.808023 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f438b3-7fe5-4987-badf-aff116693ff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:38.808412 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f438b3-7fe5-4987-badf-aff116693ff3 admin admin] PATH_INFO: `/v3/domains` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:38.809311 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f438b3-7fe5-4987-badf-aff116693ff3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:20:38.809664 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f438b3-7fe5-4987-badf-aff116693ff3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:20:38.814272 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f438b3-7fe5-4987-badf-aff116693ff3 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:20:38.824134 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:20:38.837050 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1219/2430] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Jul 26 14:20:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:20:38.842562 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f438b3-7fe5-4987-badf-aff116693ff3 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:20:38.856699 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f438b3-7fe5-4987-badf-aff116693ff3 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:20:38.865788 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d5e2a1-f0c6-424f-a81f-3876d8976d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:20:38.866638 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d5e2a1-f0c6-424f-a81f-3876d8976d6a admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:38.866920 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d5e2a1-f0c6-424f-a81f-3876d8976d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:38.867191 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d5e2a1-f0c6-424f-a81f-3876d8976d6a admin admin] PATH_INFO: `/v3/projects` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:38.867869 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d5e2a1-f0c6-424f-a81f-3876d8976d6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:20:38.869826 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d5e2a1-f0c6-424f-a81f-3876d8976d6a admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:20:38.916224 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-a60d0856-849b-477a-88f2-29b7b927b789 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75718c4f49434ff3853eddaa1209bf1b', 'request_id': 'req-a60d0856-849b-477a-88f2-29b7b927b789'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:20:38.933715 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-a60d0856-849b-477a-88f2-29b7b927b789 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75718c4f49434ff3853eddaa1209bf1b', 'request_id': 'req-a60d0856-849b-477a-88f2-29b7b927b789'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:20:38.946966 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-a60d0856-849b-477a-88f2-29b7b927b789 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75718c4f49434ff3853eddaa1209bf1b', 'request_id': 'req-a60d0856-849b-477a-88f2-29b7b927b789'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:20:38.960189 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1220/2431] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 14:20:38 2022] POST /identity/v3/projects => generated 371 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:20:38.971396 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d5e2a1-f0c6-424f-a81f-3876d8976d6a admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:20:38.988628 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-a60d0856-849b-477a-88f2-29b7b927b789 admin admin] The token cache is being invalidate because project 75718c4f49434ff3853eddaa1209bf1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:20:38.990204 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-a60d0856-849b-477a-88f2-29b7b927b789 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:20:38.992172 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-a60d0856-849b-477a-88f2-29b7b927b789 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:20:38.996146 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1212/2432] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 14:20:38 2022] DELETE /identity/v3/projects/75718c4f49434ff3853eddaa1209bf1b => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:20:39.002302 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60d0856-849b-477a-88f2-29b7b927b789 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:20:39.006408 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-a60d0856-849b-477a-88f2-29b7b927b789 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:20:39.104574 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d5e2a1-f0c6-424f-a81f-3876d8976d6a admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:20:39.111260 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e80c692-734f-4181-9027-9189f8ce25d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:20:39.112170 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e80c692-734f-4181-9027-9189f8ce25d7 admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:39.112982 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e80c692-734f-4181-9027-9189f8ce25d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:39.113583 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e80c692-734f-4181-9027-9189f8ce25d7 admin admin] PATH_INFO: `/v3/users` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:39.114284 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e80c692-734f-4181-9027-9189f8ce25d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:20:39.116519 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e80c692-734f-4181-9027-9189f8ce25d7 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:20:39.119970 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:20:39.152914 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60d0856-849b-477a-88f2-29b7b927b789 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:20:39.155383 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1221/2433] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 14:20:38 2022] POST /identity/v3/users => generated 286 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 14:20:39.159620 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e80c692-734f-4181-9027-9189f8ce25d7 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:20:39.161296 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-3e80c692-734f-4181-9027-9189f8ce25d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:20:39.164829 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c458fe6-3984-4cf8-85bc-c35e1674dcde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:20:39.165728 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c458fe6-3984-4cf8-85bc-c35e1674dcde admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:39.165973 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c458fe6-3984-4cf8-85bc-c35e1674dcde admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:39.166143 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c458fe6-3984-4cf8-85bc-c35e1674dcde admin admin] PATH_INFO: `/v3/users/c4e222a678ff4a44b74d252aa6621608` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:39.167821 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c458fe6-3984-4cf8-85bc-c35e1674dcde admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4e222a678ff4a44b74d252aa6621608)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:20:39.170446 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c458fe6-3984-4cf8-85bc-c35e1674dcde admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:20:39.179601 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e80c692-734f-4181-9027-9189f8ce25d7 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:20:39.187516 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b88cb4-400e-4458-907b-875aa42e7cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:20:39.188973 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b88cb4-400e-4458-907b-875aa42e7cc8 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:39.188973 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b88cb4-400e-4458-907b-875aa42e7cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:39.188973 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b88cb4-400e-4458-907b-875aa42e7cc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:39.189715 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b88cb4-400e-4458-907b-875aa42e7cc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:20:39.191831 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b88cb4-400e-4458-907b-875aa42e7cc8 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:20:39.207131 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1222/2434] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:20:39 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 14:20:39.212427 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b88cb4-400e-4458-907b-875aa42e7cc8 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:20:39.226908 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b88cb4-400e-4458-907b-875aa42e7cc8 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:20:39.233980 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba0bccc-cc66-421d-935f-b681576a549b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:20:39.234763 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba0bccc-cc66-421d-935f-b681576a549b admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:39.235177 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba0bccc-cc66-421d-935f-b681576a549b admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:39.235563 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba0bccc-cc66-421d-935f-b681576a549b admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:39.236384 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba0bccc-cc66-421d-935f-b681576a549b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:20:39.238190 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba0bccc-cc66-421d-935f-b681576a549b admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:20:39.251091 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1223/2435] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:20:39 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:20:39.255614 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba0bccc-cc66-421d-935f-b681576a549b admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:20:39.258195 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-1c458fe6-3984-4cf8-85bc-c35e1674dcde admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4e222a678ff4a44b74d252aa6621608'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:20:39.258921 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-1c458fe6-3984-4cf8-85bc-c35e1674dcde admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4e222a678ff4a44b74d252aa6621608'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:20:39.269860 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-1c458fe6-3984-4cf8-85bc-c35e1674dcde admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4e222a678ff4a44b74d252aa6621608'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:20:39.270679 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-1c458fe6-3984-4cf8-85bc-c35e1674dcde admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4e222a678ff4a44b74d252aa6621608'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:20:39.277072 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba0bccc-cc66-421d-935f-b681576a549b admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:20:39.292005 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c32f302-9f25-4673-8f48-cd0a2353687f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:20:39.295888 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c32f302-9f25-4673-8f48-cd0a2353687f admin admin] REQUEST_METHOD: `PUT` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:39.296723 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c32f302-9f25-4673-8f48-cd0a2353687f admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:39.297829 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c32f302-9f25-4673-8f48-cd0a2353687f admin admin] PATH_INFO: `/v3/projects/c85546b1cfa54a55896b1f28cd3b983a/users/d18cab03b71a45dc812dce21798ae806/roles/759f389b221049c4b01faf3b984cd8d7` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:39.298046 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-1c458fe6-3984-4cf8-85bc-c35e1674dcde admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4e222a678ff4a44b74d252aa6621608'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:20:39.322689 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c32f302-9f25-4673-8f48-cd0a2353687f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c85546b1cfa54a55896b1f28cd3b983a, user_id=d18cab03b71a45dc812dce21798ae806, role_id=759f389b221049c4b01faf3b984cd8d7)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:20:39.324780 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1213/2436] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 14:20:39 2022] DELETE /identity/v3/users/c4e222a678ff4a44b74d252aa6621608 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:20:39.326634 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c32f302-9f25-4673-8f48-cd0a2353687f admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:20:39.349661 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1224/2437] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 14:20:39 2022] PUT /identity/v3/projects/c85546b1cfa54a55896b1f28cd3b983a/users/d18cab03b71a45dc812dce21798ae806/roles/759f389b221049c4b01faf3b984cd8d7 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:20:39.355224 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cde85d2-04fa-45df-9ba6-37c32c4e726b None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:39.356209 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cde85d2-04fa-45df-9ba6-37c32c4e726b None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:39.356413 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cde85d2-04fa-45df-9ba6-37c32c4e726b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:39.396102 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-2cde85d2-04fa-45df-9ba6-37c32c4e726b None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:20:39.398357 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-2cde85d2-04fa-45df-9ba6-37c32c4e726b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:20:39.604700 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1214/2438] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:20:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:20:39.665886 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c32f302-9f25-4673-8f48-cd0a2353687f admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:20:39.669370 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-8c32f302-9f25-4673-8f48-cd0a2353687f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:20:39.803977 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c32f302-9f25-4673-8f48-cd0a2353687f admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:20:39.808079 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c17926d-2144-4d55-a19b-a1e79645841b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:20:39.808715 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c17926d-2144-4d55-a19b-a1e79645841b service neutron] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:39.809051 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c17926d-2144-4d55-a19b-a1e79645841b service neutron] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:39.809333 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c17926d-2144-4d55-a19b-a1e79645841b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:39.825688 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c17926d-2144-4d55-a19b-a1e79645841b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:20:39.827985 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c17926d-2144-4d55-a19b-a1e79645841b service neutron] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:20:39.836436 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cde85d2-04fa-45df-9ba6-37c32c4e726b None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:20:39.838741 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-2cde85d2-04fa-45df-9ba6-37c32c4e726b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:20:39.846264 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1225/2439] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:20:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:20:39.864252 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cde85d2-04fa-45df-9ba6-37c32c4e726b None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:20:39.874096 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fa09cb-6a52-40b3-812b-b5d1513ab6a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:20:39.876745 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fa09cb-6a52-40b3-812b-b5d1513ab6a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:39.876745 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fa09cb-6a52-40b3-812b-b5d1513ab6a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:39.876745 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fa09cb-6a52-40b3-812b-b5d1513ab6a9 admin admin] PATH_INFO: `/v3/projects/39c93df4f3a64417bb5337e3eae00cf7` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:39.878415 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fa09cb-6a52-40b3-812b-b5d1513ab6a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=39c93df4f3a64417bb5337e3eae00cf7)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:20:39.880368 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fa09cb-6a52-40b3-812b-b5d1513ab6a9 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:20:39.953500 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-e6fa09cb-6a52-40b3-812b-b5d1513ab6a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39c93df4f3a64417bb5337e3eae00cf7', 'request_id': 'req-e6fa09cb-6a52-40b3-812b-b5d1513ab6a9'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:20:39.961139 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-e6fa09cb-6a52-40b3-812b-b5d1513ab6a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39c93df4f3a64417bb5337e3eae00cf7', 'request_id': 'req-e6fa09cb-6a52-40b3-812b-b5d1513ab6a9'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:20:39.964148 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:20:39.970879 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-e6fa09cb-6a52-40b3-812b-b5d1513ab6a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39c93df4f3a64417bb5337e3eae00cf7', 'request_id': 'req-e6fa09cb-6a52-40b3-812b-b5d1513ab6a9'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:20:40.007091 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-e6fa09cb-6a52-40b3-812b-b5d1513ab6a9 admin admin] The token cache is being invalidate because project 39c93df4f3a64417bb5337e3eae00cf7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:20:40.007568 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-e6fa09cb-6a52-40b3-812b-b5d1513ab6a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:20:40.008366 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-e6fa09cb-6a52-40b3-812b-b5d1513ab6a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:20:40.010639 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1215/2440] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 14:20:39 2022] DELETE /identity/v3/projects/39c93df4f3a64417bb5337e3eae00cf7 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:20:40.020446 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a2d5d9-1895-4714-b93d-1189b398f0ae None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:40.020747 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a2d5d9-1895-4714-b93d-1189b398f0ae None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:40.021013 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a2d5d9-1895-4714-b93d-1189b398f0ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:40.043687 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-34a2d5d9-1895-4714-b93d-1189b398f0ae None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:20:40.051019 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-34a2d5d9-1895-4714-b93d-1189b398f0ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:20:40.060109 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:20:40.157700 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1226/2441] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:20:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:20:40.167580 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc4f3b1-3619-4990-b893-d92c169a1b9f None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:40.168207 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc4f3b1-3619-4990-b893-d92c169a1b9f None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:40.168545 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc4f3b1-3619-4990-b893-d92c169a1b9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:40.205072 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-9fc4f3b1-3619-4990-b893-d92c169a1b9f None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:20:40.206443 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-9fc4f3b1-3619-4990-b893-d92c169a1b9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:20:40.223529 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1216/2442] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:20:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:20:40.230501 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a2d5d9-1895-4714-b93d-1189b398f0ae None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:20:40.261868 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a2d5d9-1895-4714-b93d-1189b398f0ae None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:20:40.261868 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a8cbaa-21fe-4da0-a02f-78587ddd3d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:20:40.261868 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a8cbaa-21fe-4da0-a02f-78587ddd3d12 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:40.261868 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a8cbaa-21fe-4da0-a02f-78587ddd3d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:40.262564 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a8cbaa-21fe-4da0-a02f-78587ddd3d12 admin admin] PATH_INFO: `/v3/domains` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:40.262564 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a8cbaa-21fe-4da0-a02f-78587ddd3d12 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:20:40.262564 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a8cbaa-21fe-4da0-a02f-78587ddd3d12 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:20:40.262564 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a8cbaa-21fe-4da0-a02f-78587ddd3d12 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:20:40.275789 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1227/2443] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Jul 26 14:20:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:20:40.282557 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc4f3b1-3619-4990-b893-d92c169a1b9f None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:20:40.292983 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc4f3b1-3619-4990-b893-d92c169a1b9f None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:20:40.299207 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c245a3e0-1eda-4da4-a8a9-2d523cfdc608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:20:40.299888 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c245a3e0-1eda-4da4-a8a9-2d523cfdc608 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:40.300144 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c245a3e0-1eda-4da4-a8a9-2d523cfdc608 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:40.300388 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c245a3e0-1eda-4da4-a8a9-2d523cfdc608 admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:40.301118 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c245a3e0-1eda-4da4-a8a9-2d523cfdc608 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:20:40.303060 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c245a3e0-1eda-4da4-a8a9-2d523cfdc608 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:20:40.349536 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1217/2444] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 14:20:40 2022] POST /identity/v3/projects => generated 373 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:20:40.354559 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a8cbaa-21fe-4da0-a02f-78587ddd3d12 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:20:40.365120 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a8cbaa-21fe-4da0-a02f-78587ddd3d12 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:20:40.447560 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5393d29-7df2-4327-881a-bab3329731be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:20:40.448131 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5393d29-7df2-4327-881a-bab3329731be admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:40.448391 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5393d29-7df2-4327-881a-bab3329731be admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:40.448874 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5393d29-7df2-4327-881a-bab3329731be admin admin] PATH_INFO: `/v3/users` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:40.449787 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5393d29-7df2-4327-881a-bab3329731be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:20:40.453070 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5393d29-7df2-4327-881a-bab3329731be admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:20:40.488020 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1228/2445] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 14:20:40 2022] POST /identity/v3/users => generated 287 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:20:40.493629 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c245a3e0-1eda-4da4-a8a9-2d523cfdc608 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:20:40.509535 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c245a3e0-1eda-4da4-a8a9-2d523cfdc608 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:20:40.517170 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cde90b-2843-41b7-b3ad-a65505093801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:20:40.518064 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cde90b-2843-41b7-b3ad-a65505093801 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:40.518945 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cde90b-2843-41b7-b3ad-a65505093801 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:40.519379 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cde90b-2843-41b7-b3ad-a65505093801 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:40.520831 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cde90b-2843-41b7-b3ad-a65505093801 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:20:40.523966 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cde90b-2843-41b7-b3ad-a65505093801 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:20:40.541402 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1218/2446] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:20:40 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:20:40.546155 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5393d29-7df2-4327-881a-bab3329731be admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:20:40.560576 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5393d29-7df2-4327-881a-bab3329731be admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:20:40.566872 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd99dbaf-c0d2-402e-9147-9b8c4a93f0d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:20:40.567861 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd99dbaf-c0d2-402e-9147-9b8c4a93f0d4 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:40.569584 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd99dbaf-c0d2-402e-9147-9b8c4a93f0d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:40.569584 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd99dbaf-c0d2-402e-9147-9b8c4a93f0d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:40.569908 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd99dbaf-c0d2-402e-9147-9b8c4a93f0d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:20:40.571979 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd99dbaf-c0d2-402e-9147-9b8c4a93f0d4 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:20:40.583889 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1229/2447] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:20:40 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:20:40.588645 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cde90b-2843-41b7-b3ad-a65505093801 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:20:40.603947 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cde90b-2843-41b7-b3ad-a65505093801 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:20:40.610070 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c8d9ae-e481-417b-9de4-5d8d49581f32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:20:40.610806 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c8d9ae-e481-417b-9de4-5d8d49581f32 admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:40.611018 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c8d9ae-e481-417b-9de4-5d8d49581f32 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:40.611334 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c8d9ae-e481-417b-9de4-5d8d49581f32 admin admin] PATH_INFO: `/v3/projects/6fd7e897495a4482ae0f4ca2b7d44878/users/9b9e09a71d5e400081337b2b4e487ff5/roles/759f389b221049c4b01faf3b984cd8d7` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:40.633618 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c8d9ae-e481-417b-9de4-5d8d49581f32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fd7e897495a4482ae0f4ca2b7d44878, user_id=9b9e09a71d5e400081337b2b4e487ff5, role_id=759f389b221049c4b01faf3b984cd8d7)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:20:40.636613 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c8d9ae-e481-417b-9de4-5d8d49581f32 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:20:40.659506 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1219/2448] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 14:20:40 2022] PUT /identity/v3/projects/6fd7e897495a4482ae0f4ca2b7d44878/users/9b9e09a71d5e400081337b2b4e487ff5/roles/759f389b221049c4b01faf3b984cd8d7 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:20:40.671765 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b87b7b2-53a1-459d-961a-664f96aaa147 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:40.672652 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b87b7b2-53a1-459d-961a-664f96aaa147 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:40.673382 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b87b7b2-53a1-459d-961a-664f96aaa147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:40.694993 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-9b87b7b2-53a1-459d-961a-664f96aaa147 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:20:40.696554 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-9b87b7b2-53a1-459d-961a-664f96aaa147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:20:40.813043 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1230/2449] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:20:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:20:40.866108 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c8d9ae-e481-417b-9de4-5d8d49581f32 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:20:40.869756 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-43c8d9ae-e481-417b-9de4-5d8d49581f32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:20:40.968791 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c8d9ae-e481-417b-9de4-5d8d49581f32 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:20:40.975286 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e014de7-1589-4244-9f87-be1b4b031041 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:20:40.976185 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e014de7-1589-4244-9f87-be1b4b031041 service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:40.976669 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e014de7-1589-4244-9f87-be1b4b031041 service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:40.977125 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e014de7-1589-4244-9f87-be1b4b031041 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:40.992363 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e014de7-1589-4244-9f87-be1b4b031041 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:20:40.994146 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e014de7-1589-4244-9f87-be1b4b031041 service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:20:41.012294 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1220/2450] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:20:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:20:42.503923 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87826930-1ddf-41da-9b7d-bf3b6a363315 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:42.504919 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87826930-1ddf-41da-9b7d-bf3b6a363315 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:42.504919 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87826930-1ddf-41da-9b7d-bf3b6a363315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:42.584129 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-87826930-1ddf-41da-9b7d-bf3b6a363315 None None] MFA Rules not processed for user `9b9e09a71d5e400081337b2b4e487ff5`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:20:42.586853 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-87826930-1ddf-41da-9b7d-bf3b6a363315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:20:42.598676 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:20:42.751728 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1231/2451] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:20:42 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Jul 26 14:20:42.756826 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e014de7-1589-4244-9f87-be1b4b031041 service neutron] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:20:42.773920 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e014de7-1589-4244-9f87-be1b4b031041 service neutron] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:20:42.781824 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643c8dc3-41ec-433d-9b43-6f3306b0a0da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:20:42.783288 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643c8dc3-41ec-433d-9b43-6f3306b0a0da admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:42.783973 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643c8dc3-41ec-433d-9b43-6f3306b0a0da admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:42.784616 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643c8dc3-41ec-433d-9b43-6f3306b0a0da admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:42.786027 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643c8dc3-41ec-433d-9b43-6f3306b0a0da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:20:42.789164 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643c8dc3-41ec-433d-9b43-6f3306b0a0da admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:20:42.840108 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1221/2452] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 14:20:42 2022] POST /identity/v3/projects => generated 373 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 14:20:42.843341 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87826930-1ddf-41da-9b7d-bf3b6a363315 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:20:42.856938 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87826930-1ddf-41da-9b7d-bf3b6a363315 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:20:43.013882 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5e870a-3c2c-4393-8ca0-103203803155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:20:43.014993 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5e870a-3c2c-4393-8ca0-103203803155 admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:43.016572 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5e870a-3c2c-4393-8ca0-103203803155 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:43.018036 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5e870a-3c2c-4393-8ca0-103203803155 admin admin] PATH_INFO: `/v3/users` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:43.018969 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5e870a-3c2c-4393-8ca0-103203803155 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:20:43.021993 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5e870a-3c2c-4393-8ca0-103203803155 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:20:43.069564 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1232/2453] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 14:20:42 2022] POST /identity/v3/users => generated 293 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:20:43.073199 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643c8dc3-41ec-433d-9b43-6f3306b0a0da admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:20:43.086905 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643c8dc3-41ec-433d-9b43-6f3306b0a0da admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:20:43.094260 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caaf135-edc2-4570-85ca-72e5bba9b758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:20:43.095006 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caaf135-edc2-4570-85ca-72e5bba9b758 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:43.095377 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caaf135-edc2-4570-85ca-72e5bba9b758 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:43.095727 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caaf135-edc2-4570-85ca-72e5bba9b758 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:43.096679 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caaf135-edc2-4570-85ca-72e5bba9b758 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:20:43.098903 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caaf135-edc2-4570-85ca-72e5bba9b758 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:20:43.120913 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1222/2454] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:20:43 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 14:20:43.126321 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5e870a-3c2c-4393-8ca0-103203803155 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:20:43.144010 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5e870a-3c2c-4393-8ca0-103203803155 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:20:43.153116 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac600be2-53ac-4c6c-be11-61cff10c98a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:20:43.154832 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac600be2-53ac-4c6c-be11-61cff10c98a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:43.155283 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac600be2-53ac-4c6c-be11-61cff10c98a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:43.155935 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac600be2-53ac-4c6c-be11-61cff10c98a9 admin admin] PATH_INFO: `/v3/projects/b629fe77bda641ec841745deab06a455/users/7379682915054af28fa37e0e11578d76/roles/6e0201a9a6484f7d8b112b1eafff70bd` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:43.186655 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac600be2-53ac-4c6c-be11-61cff10c98a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b629fe77bda641ec841745deab06a455, user_id=7379682915054af28fa37e0e11578d76, role_id=6e0201a9a6484f7d8b112b1eafff70bd)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:20:43.190249 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac600be2-53ac-4c6c-be11-61cff10c98a9 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:20:43.218561 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1233/2455] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 14:20:43 2022] PUT /identity/v3/projects/b629fe77bda641ec841745deab06a455/users/7379682915054af28fa37e0e11578d76/roles/6e0201a9a6484f7d8b112b1eafff70bd => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:20:43.222512 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caaf135-edc2-4570-85ca-72e5bba9b758 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:20:43.234699 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caaf135-edc2-4570-85ca-72e5bba9b758 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:20:43.317083 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:20:43.337137 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2bfae8-7348-4adb-b672-215abe712d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:20:43.337859 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2bfae8-7348-4adb-b672-215abe712d38 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:43.338153 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2bfae8-7348-4adb-b672-215abe712d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:43.338430 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2bfae8-7348-4adb-b672-215abe712d38 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:43.339126 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2bfae8-7348-4adb-b672-215abe712d38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:20:43.340741 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2bfae8-7348-4adb-b672-215abe712d38 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:20:43.353313 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1223/2456] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:20:43 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:20:43.358072 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac600be2-53ac-4c6c-be11-61cff10c98a9 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:20:43.369839 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac600be2-53ac-4c6c-be11-61cff10c98a9 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:20:43.374606 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d4e480-34fb-4e33-a566-0bfb86844d1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:20:43.375309 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d4e480-34fb-4e33-a566-0bfb86844d1a admin admin] REQUEST_METHOD: `PUT` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:43.375586 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d4e480-34fb-4e33-a566-0bfb86844d1a admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:43.375857 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d4e480-34fb-4e33-a566-0bfb86844d1a admin admin] PATH_INFO: `/v3/projects/b629fe77bda641ec841745deab06a455/users/7379682915054af28fa37e0e11578d76/roles/6e0201a9a6484f7d8b112b1eafff70bd` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:43.377828 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d4e480-34fb-4e33-a566-0bfb86844d1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b629fe77bda641ec841745deab06a455, user_id=7379682915054af28fa37e0e11578d76, role_id=6e0201a9a6484f7d8b112b1eafff70bd)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:20:43.380969 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d4e480-34fb-4e33-a566-0bfb86844d1a admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:20:43.393290 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1234/2457] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 14:20:43 2022] PUT /identity/v3/projects/b629fe77bda641ec841745deab06a455/users/7379682915054af28fa37e0e11578d76/roles/6e0201a9a6484f7d8b112b1eafff70bd => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:20:44.291853 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f2485d-e1d5-49f1-8fb1-eff0c7975601 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:44.293066 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f2485d-e1d5-49f1-8fb1-eff0c7975601 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:44.293258 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f2485d-e1d5-49f1-8fb1-eff0c7975601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:44.347310 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-72f2485d-e1d5-49f1-8fb1-eff0c7975601 None None] MFA Rules not processed for user `7379682915054af28fa37e0e11578d76`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:20:44.349591 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-72f2485d-e1d5-49f1-8fb1-eff0c7975601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:20:44.533357 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1224/2458] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:20:44 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:20:44.550111 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d4e480-34fb-4e33-a566-0bfb86844d1a admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:20:44.572479 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-81d4e480-34fb-4e33-a566-0bfb86844d1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:20:44.664707 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d4e480-34fb-4e33-a566-0bfb86844d1a admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:20:44.668782 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c318db7-d8dd-438e-af32-f4e450dd1970 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:20:44.669423 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c318db7-d8dd-438e-af32-f4e450dd1970 service cinder] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:44.670315 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c318db7-d8dd-438e-af32-f4e450dd1970 service cinder] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:44.670597 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c318db7-d8dd-438e-af32-f4e450dd1970 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:44.692685 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:20:44.769109 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c318db7-d8dd-438e-af32-f4e450dd1970 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:20:44.770913 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c318db7-d8dd-438e-af32-f4e450dd1970 service cinder] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:20:44.791054 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1235/2459] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:20:44 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 14:20:46.053699 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e484e4-4ea1-4b13-9319-dc35b7a743d8 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:46.054601 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e484e4-4ea1-4b13-9319-dc35b7a743d8 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:46.055204 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e484e4-4ea1-4b13-9319-dc35b7a743d8 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:46.058353 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1225/2460] 10.222.0.58 () {54 vars in 921 bytes} [Tue Jul 26 14:20:46 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:20:46.067753 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730710d3-c252-4a56-a7c5-0d1507e1201f None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:46.068688 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730710d3-c252-4a56-a7c5-0d1507e1201f None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:46.068688 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730710d3-c252-4a56-a7c5-0d1507e1201f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:46.120229 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e484e4-4ea1-4b13-9319-dc35b7a743d8 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:20:46.123688 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-29e484e4-4ea1-4b13-9319-dc35b7a743d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:20:46.156989 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-730710d3-c252-4a56-a7c5-0d1507e1201f None None] MFA Rules not processed for user `835cbc13756b41b598142df3e35f5492`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:20:46.158228 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-730710d3-c252-4a56-a7c5-0d1507e1201f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:20:46.281550 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e484e4-4ea1-4b13-9319-dc35b7a743d8 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:20:46.282009 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1236/2461] 10.222.0.58 () {58 vars in 1018 bytes} [Tue Jul 26 14:20:46 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:20:46.290946 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4662afd0-2036-4293-bc8e-93572f156938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:20:46.292101 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4662afd0-2036-4293-bc8e-93572f156938 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:46.292585 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4662afd0-2036-4293-bc8e-93572f156938 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:46.293204 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4662afd0-2036-4293-bc8e-93572f156938 admin admin] PATH_INFO: `/v3/users/1747d40a285942eb9b4d5bc14136d88a` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:46.296563 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730710d3-c252-4a56-a7c5-0d1507e1201f None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:20:46.296828 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4662afd0-2036-4293-bc8e-93572f156938 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1747d40a285942eb9b4d5bc14136d88a)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:20:46.298448 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-730710d3-c252-4a56-a7c5-0d1507e1201f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:20:46.300775 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4662afd0-2036-4293-bc8e-93572f156938 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:20:46.318467 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730710d3-c252-4a56-a7c5-0d1507e1201f None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:20:46.325929 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d50e26-8e19-4096-971f-a68142ded2a8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '835cbc13756b41b598142df3e35f5492', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:20:46.326629 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d50e26-8e19-4096-971f-a68142ded2a8 service placement] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:46.326922 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d50e26-8e19-4096-971f-a68142ded2a8 service placement] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:46.327202 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d50e26-8e19-4096-971f-a68142ded2a8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:46.344098 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d50e26-8e19-4096-971f-a68142ded2a8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:20:46.346043 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d50e26-8e19-4096-971f-a68142ded2a8 service placement] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:20:46.367696 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1237/2462] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:20:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 14:20:46.386849 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-4662afd0-2036-4293-bc8e-93572f156938 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1747d40a285942eb9b4d5bc14136d88a'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:20:46.388327 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-4662afd0-2036-4293-bc8e-93572f156938 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1747d40a285942eb9b4d5bc14136d88a'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:20:46.403431 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-4662afd0-2036-4293-bc8e-93572f156938 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1747d40a285942eb9b4d5bc14136d88a'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:20:46.405693 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-4662afd0-2036-4293-bc8e-93572f156938 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1747d40a285942eb9b4d5bc14136d88a'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:20:46.426922 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-4662afd0-2036-4293-bc8e-93572f156938 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1747d40a285942eb9b4d5bc14136d88a'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:20:46.432502 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:20:46.455717 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1226/2463] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 14:20:46 2022] DELETE /identity/v3/users/1747d40a285942eb9b4d5bc14136d88a => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:20:46.483805 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d50e26-8e19-4096-971f-a68142ded2a8 service placement] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:20:46.485373 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-f5d50e26-8e19-4096-971f-a68142ded2a8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:20:46.504476 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d50e26-8e19-4096-971f-a68142ded2a8 service placement] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:20:46.580764 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34b76e4-6d0f-437a-a959-f15a9a639f54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:20:46.581430 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34b76e4-6d0f-437a-a959-f15a9a639f54 service cinder] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:46.581793 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34b76e4-6d0f-437a-a959-f15a9a639f54 service cinder] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:46.582129 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34b76e4-6d0f-437a-a959-f15a9a639f54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:46.584219 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-e34b76e4-6d0f-437a-a959-f15a9a639f54 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:20:46.659735 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:20:46.706628 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4662afd0-2036-4293-bc8e-93572f156938 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:20:46.708335 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-4662afd0-2036-4293-bc8e-93572f156938 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:20:46.747186 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34b76e4-6d0f-437a-a959-f15a9a639f54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:20:46.748752 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34b76e4-6d0f-437a-a959-f15a9a639f54 service cinder] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:20:46.771006 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1238/2464] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:20:46 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:20:46.869033 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4662afd0-2036-4293-bc8e-93572f156938 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:20:46.906537 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6482f4-c288-4dda-911f-7137b4090a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:20:46.907490 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6482f4-c288-4dda-911f-7137b4090a64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:46.914324 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6482f4-c288-4dda-911f-7137b4090a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:46.916925 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6482f4-c288-4dda-911f-7137b4090a64 admin admin] PATH_INFO: `/v3/projects/e4a79afb56fa40598a8891c1fad1bd62` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:46.975164 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6482f4-c288-4dda-911f-7137b4090a64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4a79afb56fa40598a8891c1fad1bd62)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:20:46.980684 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6482f4-c288-4dda-911f-7137b4090a64 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:20:47.102180 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-4e6482f4-c288-4dda-911f-7137b4090a64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4a79afb56fa40598a8891c1fad1bd62', 'request_id': 'req-4e6482f4-c288-4dda-911f-7137b4090a64'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:20:47.111063 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-4e6482f4-c288-4dda-911f-7137b4090a64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4a79afb56fa40598a8891c1fad1bd62', 'request_id': 'req-4e6482f4-c288-4dda-911f-7137b4090a64'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:20:47.121883 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-4e6482f4-c288-4dda-911f-7137b4090a64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4a79afb56fa40598a8891c1fad1bd62', 'request_id': 'req-4e6482f4-c288-4dda-911f-7137b4090a64'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:20:47.141891 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-4e6482f4-c288-4dda-911f-7137b4090a64 admin admin] The token cache is being invalidate because project e4a79afb56fa40598a8891c1fad1bd62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:20:47.142368 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-4e6482f4-c288-4dda-911f-7137b4090a64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:20:47.143060 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-4e6482f4-c288-4dda-911f-7137b4090a64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:20:47.144861 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1227/2465] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 14:20:46 2022] DELETE /identity/v3/projects/e4a79afb56fa40598a8891c1fad1bd62 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:20:47.151171 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164be76f-ead9-4a29-bdd7-a84b33955607 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:47.153551 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164be76f-ead9-4a29-bdd7-a84b33955607 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:47.153551 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164be76f-ead9-4a29-bdd7-a84b33955607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:47.173436 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-164be76f-ead9-4a29-bdd7-a84b33955607 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:20:47.174744 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-164be76f-ead9-4a29-bdd7-a84b33955607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:20:47.270527 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1239/2466] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:20:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:20:47.279840 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f0aa50-afac-459e-aa61-843176c95c9b None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:47.280276 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f0aa50-afac-459e-aa61-843176c95c9b None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:47.280276 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f0aa50-afac-459e-aa61-843176c95c9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:47.315698 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-58f0aa50-afac-459e-aa61-843176c95c9b None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:20:47.317816 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-58f0aa50-afac-459e-aa61-843176c95c9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:20:47.333988 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164be76f-ead9-4a29-bdd7-a84b33955607 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:20:47.340741 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1228/2467] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:20:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:20:47.348552 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164be76f-ead9-4a29-bdd7-a84b33955607 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:20:47.355052 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a201e6de-711f-4ea8-a3d3-9002ac539c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:20:47.356206 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a201e6de-711f-4ea8-a3d3-9002ac539c27 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:47.356666 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a201e6de-711f-4ea8-a3d3-9002ac539c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:47.357350 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a201e6de-711f-4ea8-a3d3-9002ac539c27 admin admin] PATH_INFO: `/v3/domains` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:47.358274 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a201e6de-711f-4ea8-a3d3-9002ac539c27 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:20:47.358647 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a201e6de-711f-4ea8-a3d3-9002ac539c27 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:20:47.360343 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a201e6de-711f-4ea8-a3d3-9002ac539c27 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:20:47.377126 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1240/2468] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Jul 26 14:20:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:20:47.378406 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f0aa50-afac-459e-aa61-843176c95c9b None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:20:47.389807 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f0aa50-afac-459e-aa61-843176c95c9b None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:20:47.396299 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e999c4-90c0-4bb2-aee8-7a1415ee45b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:20:47.397039 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e999c4-90c0-4bb2-aee8-7a1415ee45b6 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:47.397324 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e999c4-90c0-4bb2-aee8-7a1415ee45b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:47.397635 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e999c4-90c0-4bb2-aee8-7a1415ee45b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:47.398498 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e999c4-90c0-4bb2-aee8-7a1415ee45b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:20:47.400671 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e999c4-90c0-4bb2-aee8-7a1415ee45b6 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:20:47.441038 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1229/2469] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 14:20:47 2022] POST /identity/v3/projects => generated 381 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:20:47.442558 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a201e6de-711f-4ea8-a3d3-9002ac539c27 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:20:47.453954 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a201e6de-711f-4ea8-a3d3-9002ac539c27 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:20:47.547141 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d078b87-7a62-463e-91a3-3b0423447ec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:20:47.547788 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d078b87-7a62-463e-91a3-3b0423447ec6 admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:47.548069 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d078b87-7a62-463e-91a3-3b0423447ec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:47.548341 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d078b87-7a62-463e-91a3-3b0423447ec6 admin admin] PATH_INFO: `/v3/users` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:47.549140 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d078b87-7a62-463e-91a3-3b0423447ec6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:20:47.551123 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d078b87-7a62-463e-91a3-3b0423447ec6 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:20:47.594442 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1241/2470] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 14:20:47 2022] POST /identity/v3/users => generated 291 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:20:47.599301 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e999c4-90c0-4bb2-aee8-7a1415ee45b6 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:20:47.616280 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e999c4-90c0-4bb2-aee8-7a1415ee45b6 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:20:47.630593 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a816484-7bf9-4d98-a39b-40df73423a01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:20:47.631670 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a816484-7bf9-4d98-a39b-40df73423a01 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:47.631963 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a816484-7bf9-4d98-a39b-40df73423a01 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:47.632256 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a816484-7bf9-4d98-a39b-40df73423a01 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:47.632982 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a816484-7bf9-4d98-a39b-40df73423a01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:20:47.636011 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a816484-7bf9-4d98-a39b-40df73423a01 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:20:47.650277 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1230/2471] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:20:47 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:20:47.654846 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d078b87-7a62-463e-91a3-3b0423447ec6 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:20:47.671173 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d078b87-7a62-463e-91a3-3b0423447ec6 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:20:47.678230 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4f4ed3-44ab-4e39-8303-5a4f35b4238d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:20:47.679458 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4f4ed3-44ab-4e39-8303-5a4f35b4238d admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:47.679458 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4f4ed3-44ab-4e39-8303-5a4f35b4238d admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:47.679458 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4f4ed3-44ab-4e39-8303-5a4f35b4238d admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:47.680650 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4f4ed3-44ab-4e39-8303-5a4f35b4238d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:20:47.682709 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4f4ed3-44ab-4e39-8303-5a4f35b4238d admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:20:47.707481 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1242/2472] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:20:47 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 14:20:47.708876 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a816484-7bf9-4d98-a39b-40df73423a01 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:20:47.720311 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a816484-7bf9-4d98-a39b-40df73423a01 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:20:47.725233 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362d5e2a-4d85-47c3-84d3-04f2a26bb6cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:20:47.726018 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362d5e2a-4d85-47c3-84d3-04f2a26bb6cf admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:47.726299 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362d5e2a-4d85-47c3-84d3-04f2a26bb6cf admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:47.726577 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362d5e2a-4d85-47c3-84d3-04f2a26bb6cf admin admin] PATH_INFO: `/v3/projects/94e84cb83d6e40078c1b11ad493e6bbf/users/b5f2aceab29f4803b9d72bc040a5c573/roles/759f389b221049c4b01faf3b984cd8d7` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:47.744758 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362d5e2a-4d85-47c3-84d3-04f2a26bb6cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=94e84cb83d6e40078c1b11ad493e6bbf, user_id=b5f2aceab29f4803b9d72bc040a5c573, role_id=759f389b221049c4b01faf3b984cd8d7)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:20:47.747738 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362d5e2a-4d85-47c3-84d3-04f2a26bb6cf admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:20:47.769504 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1231/2473] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 14:20:47 2022] PUT /identity/v3/projects/94e84cb83d6e40078c1b11ad493e6bbf/users/b5f2aceab29f4803b9d72bc040a5c573/roles/759f389b221049c4b01faf3b984cd8d7 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:20:47.771042 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf77610a-3fcd-42a7-b691-8c72450e9f0d None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:47.771526 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf77610a-3fcd-42a7-b691-8c72450e9f0d None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:47.771937 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf77610a-3fcd-42a7-b691-8c72450e9f0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:47.773864 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79763f93-514b-4124-a0f2-54bb0d7c89d6 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:47.774396 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79763f93-514b-4124-a0f2-54bb0d7c89d6 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:47.774850 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79763f93-514b-4124-a0f2-54bb0d7c89d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:47.821641 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:INSERT=4,keystone:DELETE=8,keystone:SELECT=88 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:20:47.834338 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-cf77610a-3fcd-42a7-b691-8c72450e9f0d None None] MFA Rules not processed for user `d18cab03b71a45dc812dce21798ae806`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:20:47.835880 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-cf77610a-3fcd-42a7-b691-8c72450e9f0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:20:47.840455 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-79763f93-514b-4124-a0f2-54bb0d7c89d6 None None] MFA Rules not processed for user `b5f2aceab29f4803b9d72bc040a5c573`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:20:47.842787 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-79763f93-514b-4124-a0f2-54bb0d7c89d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:20:47.879466 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:20:47.948752 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1232/2474] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:20:47 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:20:47.959752 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79763f93-514b-4124-a0f2-54bb0d7c89d6 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:20:47.960314 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1243/2475] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:20:47 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:20:47.964484 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf77610a-3fcd-42a7-b691-8c72450e9f0d None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:20:47.966298 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-cf77610a-3fcd-42a7-b691-8c72450e9f0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:20:48.057527 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79763f93-514b-4124-a0f2-54bb0d7c89d6 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:20:48.062676 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7c9147-349b-4289-824d-6715bbda8c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:20:48.063418 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7c9147-349b-4289-824d-6715bbda8c3e admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:48.063801 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7c9147-349b-4289-824d-6715bbda8c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:48.064173 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7c9147-349b-4289-824d-6715bbda8c3e admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:48.064959 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7c9147-349b-4289-824d-6715bbda8c3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:20:48.067037 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7c9147-349b-4289-824d-6715bbda8c3e admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:20:48.088083 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf77610a-3fcd-42a7-b691-8c72450e9f0d None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:20:48.096962 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1285ef09-5b83-4291-9eab-676abb3a2414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:20:48.098075 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1285ef09-5b83-4291-9eab-676abb3a2414 admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:48.098531 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1285ef09-5b83-4291-9eab-676abb3a2414 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:48.099020 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1285ef09-5b83-4291-9eab-676abb3a2414 admin admin] PATH_INFO: `/v3/projects` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:48.100314 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1285ef09-5b83-4291-9eab-676abb3a2414 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:20:48.103937 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1233/2476] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 14:20:47 2022] POST /identity/v3/projects => generated 383 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:20:48.104437 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1285ef09-5b83-4291-9eab-676abb3a2414 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:20:48.111044 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7c9147-349b-4289-824d-6715bbda8c3e admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:20:48.152270 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1244/2477] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 14:20:47 2022] POST /identity/v3/projects => generated 373 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:20:48.157312 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1285ef09-5b83-4291-9eab-676abb3a2414 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:20:48.227363 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7c9147-349b-4289-824d-6715bbda8c3e admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:20:48.250331 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1285ef09-5b83-4291-9eab-676abb3a2414 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:20:48.257662 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4e2c3b-e85d-46bb-9854-64dc6441ece1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:20:48.258603 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4e2c3b-e85d-46bb-9854-64dc6441ece1 admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:48.259062 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4e2c3b-e85d-46bb-9854-64dc6441ece1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:48.259511 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4e2c3b-e85d-46bb-9854-64dc6441ece1 admin admin] PATH_INFO: `/v3/users` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:48.260353 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4e2c3b-e85d-46bb-9854-64dc6441ece1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:20:48.262579 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4e2c3b-e85d-46bb-9854-64dc6441ece1 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:20:48.270850 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:20:48.290980 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1245/2478] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 14:20:48 2022] POST /identity/v3/users => generated 293 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 14:20:48.295480 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4e2c3b-e85d-46bb-9854-64dc6441ece1 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:20:48.309826 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f37a11-1b72-4804-aa1c-eeb55bc2053f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:20:48.311137 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f37a11-1b72-4804-aa1c-eeb55bc2053f admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:48.311479 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f37a11-1b72-4804-aa1c-eeb55bc2053f admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:48.311761 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f37a11-1b72-4804-aa1c-eeb55bc2053f admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:48.312404 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f37a11-1b72-4804-aa1c-eeb55bc2053f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:20:48.314838 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f37a11-1b72-4804-aa1c-eeb55bc2053f admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:20:48.317886 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4e2c3b-e85d-46bb-9854-64dc6441ece1 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:20:48.326758 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a824bdda-ac35-447b-a630-5094fb51c90a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:20:48.328320 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a824bdda-ac35-447b-a630-5094fb51c90a admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:48.328954 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a824bdda-ac35-447b-a630-5094fb51c90a admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:48.329491 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a824bdda-ac35-447b-a630-5094fb51c90a admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:48.330671 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a824bdda-ac35-447b-a630-5094fb51c90a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:20:48.332778 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a824bdda-ac35-447b-a630-5094fb51c90a admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:20:48.345290 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1234/2479] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 14:20:48 2022] POST /identity/v3/users => generated 298 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 14:20:48.351528 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1246/2480] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:20:48 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:20:48.351720 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f37a11-1b72-4804-aa1c-eeb55bc2053f admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:20:48.352993 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a824bdda-ac35-447b-a630-5094fb51c90a admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:20:48.365311 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f37a11-1b72-4804-aa1c-eeb55bc2053f admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:20:48.365856 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a824bdda-ac35-447b-a630-5094fb51c90a admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:20:48.371729 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a9a341-1d6d-478a-a6e7-42d155ce851e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:20:48.373573 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb8ab72-81c8-47aa-afa6-2f17160228f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:20:48.374111 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a9a341-1d6d-478a-a6e7-42d155ce851e admin admin] REQUEST_METHOD: `PUT` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:48.374503 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb8ab72-81c8-47aa-afa6-2f17160228f4 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:48.374883 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb8ab72-81c8-47aa-afa6-2f17160228f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:48.375084 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a9a341-1d6d-478a-a6e7-42d155ce851e admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:48.375235 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb8ab72-81c8-47aa-afa6-2f17160228f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:48.375746 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a9a341-1d6d-478a-a6e7-42d155ce851e admin admin] PATH_INFO: `/v3/projects/426041dec62c40db96d4a4c4f0cde3c7/users/83d3ef25325c4fc6a0b302d459c4c617/roles/6e0201a9a6484f7d8b112b1eafff70bd` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:48.375879 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb8ab72-81c8-47aa-afa6-2f17160228f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:20:48.377838 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb8ab72-81c8-47aa-afa6-2f17160228f4 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:20:48.401253 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1235/2481] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:20:48 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:20:48.406778 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb8ab72-81c8-47aa-afa6-2f17160228f4 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:20:48.417959 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a9a341-1d6d-478a-a6e7-42d155ce851e admin admin] RBAC: Authorizing `identity:create_grant(project_id=426041dec62c40db96d4a4c4f0cde3c7, user_id=83d3ef25325c4fc6a0b302d459c4c617, role_id=6e0201a9a6484f7d8b112b1eafff70bd)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:20:48.424066 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a9a341-1d6d-478a-a6e7-42d155ce851e admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:20:48.449270 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb8ab72-81c8-47aa-afa6-2f17160228f4 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:20:48.468527 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:20:48.469658 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4befb261-b0d9-4ecb-895a-33c765875ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:20:48.471333 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4befb261-b0d9-4ecb-895a-33c765875ca8 admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:48.471333 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4befb261-b0d9-4ecb-895a-33c765875ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:48.471333 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4befb261-b0d9-4ecb-895a-33c765875ca8 admin admin] PATH_INFO: `/v3/projects/c4ccd9e2d524414eaece4b51aaf4692d/users/42cdc69f51c2497bb765c87b19314166/roles/6e0201a9a6484f7d8b112b1eafff70bd` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:48.481310 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1247/2482] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 14:20:48 2022] PUT /identity/v3/projects/426041dec62c40db96d4a4c4f0cde3c7/users/83d3ef25325c4fc6a0b302d459c4c617/roles/6e0201a9a6484f7d8b112b1eafff70bd => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:20:48.484570 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a9a341-1d6d-478a-a6e7-42d155ce851e admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:20:48.497637 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4befb261-b0d9-4ecb-895a-33c765875ca8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4ccd9e2d524414eaece4b51aaf4692d, user_id=42cdc69f51c2497bb765c87b19314166, role_id=6e0201a9a6484f7d8b112b1eafff70bd)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:20:48.500767 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4befb261-b0d9-4ecb-895a-33c765875ca8 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:20:48.532596 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1236/2483] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 14:20:48 2022] PUT /identity/v3/projects/c4ccd9e2d524414eaece4b51aaf4692d/users/42cdc69f51c2497bb765c87b19314166/roles/6e0201a9a6484f7d8b112b1eafff70bd => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:20:48.536543 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4befb261-b0d9-4ecb-895a-33c765875ca8 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:20:48.584339 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a9a341-1d6d-478a-a6e7-42d155ce851e admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:20:48.630637 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4befb261-b0d9-4ecb-895a-33c765875ca8 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:20:48.639458 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf85694f-3416-40c1-a564-779cbda99077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:20:48.642174 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf85694f-3416-40c1-a564-779cbda99077 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:48.642174 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf85694f-3416-40c1-a564-779cbda99077 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:48.649917 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf85694f-3416-40c1-a564-779cbda99077 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:48.651492 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf85694f-3416-40c1-a564-779cbda99077 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:20:48.653149 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf85694f-3416-40c1-a564-779cbda99077 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:20:48.666088 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1237/2484] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:20:48 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 14:20:48.670068 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf85694f-3416-40c1-a564-779cbda99077 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:20:48.681690 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf85694f-3416-40c1-a564-779cbda99077 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:20:48.689236 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fe780e-2348-4f88-a37a-f9bcb462fbc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:20:48.689973 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fe780e-2348-4f88-a37a-f9bcb462fbc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:48.690286 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fe780e-2348-4f88-a37a-f9bcb462fbc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:48.690577 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fe780e-2348-4f88-a37a-f9bcb462fbc8 admin admin] PATH_INFO: `/v3/projects/c4ccd9e2d524414eaece4b51aaf4692d/users/42cdc69f51c2497bb765c87b19314166/roles/6e0201a9a6484f7d8b112b1eafff70bd` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:48.692902 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fe780e-2348-4f88-a37a-f9bcb462fbc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4ccd9e2d524414eaece4b51aaf4692d, user_id=42cdc69f51c2497bb765c87b19314166, role_id=6e0201a9a6484f7d8b112b1eafff70bd)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:20:48.695944 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fe780e-2348-4f88-a37a-f9bcb462fbc8 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:20:48.714661 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1238/2485] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 14:20:48 2022] PUT /identity/v3/projects/c4ccd9e2d524414eaece4b51aaf4692d/users/42cdc69f51c2497bb765c87b19314166/roles/6e0201a9a6484f7d8b112b1eafff70bd => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:20:48.727607 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d452cfd-57aa-41b2-af5b-1c72099fab7b None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:48.727907 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d452cfd-57aa-41b2-af5b-1c72099fab7b None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:48.728181 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d452cfd-57aa-41b2-af5b-1c72099fab7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:48.750892 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7317c5d4-04cd-441e-a8ea-0bfec4d57296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:20:48.751984 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7317c5d4-04cd-441e-a8ea-0bfec4d57296 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:48.752302 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7317c5d4-04cd-441e-a8ea-0bfec4d57296 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:48.752657 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7317c5d4-04cd-441e-a8ea-0bfec4d57296 admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:48.753528 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7317c5d4-04cd-441e-a8ea-0bfec4d57296 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:20:48.756890 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7317c5d4-04cd-441e-a8ea-0bfec4d57296 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:20:48.765292 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-3d452cfd-57aa-41b2-af5b-1c72099fab7b None None] MFA Rules not processed for user `42cdc69f51c2497bb765c87b19314166`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:20:48.766579 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-3d452cfd-57aa-41b2-af5b-1c72099fab7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:20:48.773250 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1248/2486] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:20:48 2022] GET /identity/v3/roles => generated 1420 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:20:48.775167 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7317c5d4-04cd-441e-a8ea-0bfec4d57296 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:20:48.791187 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7317c5d4-04cd-441e-a8ea-0bfec4d57296 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:20:48.846469 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:20:48.864742 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1239/2487] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:20:48 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:20:48.867171 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8467f034-0870-4f7e-991e-60a57649ffb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:20:48.868160 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8467f034-0870-4f7e-991e-60a57649ffb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:48.868800 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8467f034-0870-4f7e-991e-60a57649ffb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:48.869880 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8467f034-0870-4f7e-991e-60a57649ffb0 admin admin] PATH_INFO: `/v3/projects/426041dec62c40db96d4a4c4f0cde3c7/users/83d3ef25325c4fc6a0b302d459c4c617/roles/6e0201a9a6484f7d8b112b1eafff70bd` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:48.872415 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8467f034-0870-4f7e-991e-60a57649ffb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=426041dec62c40db96d4a4c4f0cde3c7, user_id=83d3ef25325c4fc6a0b302d459c4c617, role_id=6e0201a9a6484f7d8b112b1eafff70bd)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:20:48.875491 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8467f034-0870-4f7e-991e-60a57649ffb0 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:20:48.879994 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d452cfd-57aa-41b2-af5b-1c72099fab7b None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:20:48.883045 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-3d452cfd-57aa-41b2-af5b-1c72099fab7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:20:48.893086 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1249/2488] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 14:20:48 2022] PUT /identity/v3/projects/426041dec62c40db96d4a4c4f0cde3c7/users/83d3ef25325c4fc6a0b302d459c4c617/roles/6e0201a9a6484f7d8b112b1eafff70bd => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:20:48.960055 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d452cfd-57aa-41b2-af5b-1c72099fab7b None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:20:49.064704 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07710e5-b5ef-48d7-8c85-0e09fba41d30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:20:49.065336 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07710e5-b5ef-48d7-8c85-0e09fba41d30 service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:49.065682 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07710e5-b5ef-48d7-8c85-0e09fba41d30 service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:49.065977 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07710e5-b5ef-48d7-8c85-0e09fba41d30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:49.153492 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07710e5-b5ef-48d7-8c85-0e09fba41d30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:20:49.155821 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07710e5-b5ef-48d7-8c85-0e09fba41d30 service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:20:49.170778 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1240/2489] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:20:48 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:20:49.268393 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8467f034-0870-4f7e-991e-60a57649ffb0 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:20:49.272233 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-8467f034-0870-4f7e-991e-60a57649ffb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:20:49.291116 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8467f034-0870-4f7e-991e-60a57649ffb0 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:20:49.295934 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8539f9d3-bc92-48b8-bde1-6dea153814fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:20:49.296753 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8539f9d3-bc92-48b8-bde1-6dea153814fb service cinder] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:49.297182 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8539f9d3-bc92-48b8-bde1-6dea153814fb service cinder] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:49.297588 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8539f9d3-bc92-48b8-bde1-6dea153814fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:49.344384 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:20:49.402170 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8539f9d3-bc92-48b8-bde1-6dea153814fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:20:49.404867 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8539f9d3-bc92-48b8-bde1-6dea153814fb service cinder] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:20:49.427012 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1250/2490] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:20:49 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:20:49.892593 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07710e5-b5ef-48d7-8c85-0e09fba41d30 service cinder] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:20:50.045278 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07710e5-b5ef-48d7-8c85-0e09fba41d30 service cinder] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:20:50.059784 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84c771d-13e1-4049-8690-5c7c3561b210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:20:50.060835 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84c771d-13e1-4049-8690-5c7c3561b210 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:50.061325 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84c771d-13e1-4049-8690-5c7c3561b210 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:50.061786 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84c771d-13e1-4049-8690-5c7c3561b210 admin admin] PATH_INFO: `/v3/users/b5f2aceab29f4803b9d72bc040a5c573` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:50.065302 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84c771d-13e1-4049-8690-5c7c3561b210 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5f2aceab29f4803b9d72bc040a5c573)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:20:50.068378 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84c771d-13e1-4049-8690-5c7c3561b210 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:20:50.072218 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:20:50.171782 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-d84c771d-13e1-4049-8690-5c7c3561b210 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5f2aceab29f4803b9d72bc040a5c573'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:20:50.173175 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-d84c771d-13e1-4049-8690-5c7c3561b210 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5f2aceab29f4803b9d72bc040a5c573'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:20:50.195592 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-d84c771d-13e1-4049-8690-5c7c3561b210 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5f2aceab29f4803b9d72bc040a5c573'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:20:50.197795 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-d84c771d-13e1-4049-8690-5c7c3561b210 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5f2aceab29f4803b9d72bc040a5c573'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:20:50.261689 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-d84c771d-13e1-4049-8690-5c7c3561b210 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5f2aceab29f4803b9d72bc040a5c573'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:20:50.304565 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1241/2491] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 14:20:49 2022] DELETE /identity/v3/users/b5f2aceab29f4803b9d72bc040a5c573 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:20:50.310282 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41f563c-973d-41fa-9b0a-23ab9c590dab None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:50.310840 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41f563c-973d-41fa-9b0a-23ab9c590dab None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:50.310950 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41f563c-973d-41fa-9b0a-23ab9c590dab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:50.349351 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-e41f563c-973d-41fa-9b0a-23ab9c590dab None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:20:50.350625 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-e41f563c-973d-41fa-9b0a-23ab9c590dab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:20:50.496197 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1251/2492] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:20:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:20:50.512562 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84c771d-13e1-4049-8690-5c7c3561b210 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:20:50.514929 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-d84c771d-13e1-4049-8690-5c7c3561b210 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:20:50.695617 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84c771d-13e1-4049-8690-5c7c3561b210 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:20:50.705788 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d65b1c8-96f3-477f-9fae-503fa29b0b99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:20:50.706735 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d65b1c8-96f3-477f-9fae-503fa29b0b99 service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:50.707006 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d65b1c8-96f3-477f-9fae-503fa29b0b99 service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:50.707314 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d65b1c8-96f3-477f-9fae-503fa29b0b99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:50.735672 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d65b1c8-96f3-477f-9fae-503fa29b0b99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:20:50.738390 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d65b1c8-96f3-477f-9fae-503fa29b0b99 service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:20:50.767985 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1242/2493] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:20:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 14:20:51.469959 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41f563c-973d-41fa-9b0a-23ab9c590dab None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:20:51.472822 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-e41f563c-973d-41fa-9b0a-23ab9c590dab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:20:51.502007 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41f563c-973d-41fa-9b0a-23ab9c590dab None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:20:51.515956 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57784888-834a-4159-b227-97076d992b36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:20:51.517576 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57784888-834a-4159-b227-97076d992b36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:51.518003 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57784888-834a-4159-b227-97076d992b36 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:51.518284 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57784888-834a-4159-b227-97076d992b36 admin admin] PATH_INFO: `/v3/projects/94e84cb83d6e40078c1b11ad493e6bbf` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:51.520203 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57784888-834a-4159-b227-97076d992b36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94e84cb83d6e40078c1b11ad493e6bbf)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:20:51.522432 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57784888-834a-4159-b227-97076d992b36 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:20:51.599434 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-57784888-834a-4159-b227-97076d992b36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94e84cb83d6e40078c1b11ad493e6bbf', 'request_id': 'req-57784888-834a-4159-b227-97076d992b36'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:20:51.611658 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-57784888-834a-4159-b227-97076d992b36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94e84cb83d6e40078c1b11ad493e6bbf', 'request_id': 'req-57784888-834a-4159-b227-97076d992b36'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:20:51.627720 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-57784888-834a-4159-b227-97076d992b36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94e84cb83d6e40078c1b11ad493e6bbf', 'request_id': 'req-57784888-834a-4159-b227-97076d992b36'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:20:51.670050 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-57784888-834a-4159-b227-97076d992b36 admin admin] The token cache is being invalidate because project 94e84cb83d6e40078c1b11ad493e6bbf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86933) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:20:51.670233 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-57784888-834a-4159-b227-97076d992b36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:20:51.672986 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-57784888-834a-4159-b227-97076d992b36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:20:51.680096 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1252/2494] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 14:20:51 2022] DELETE /identity/v3/projects/94e84cb83d6e40078c1b11ad493e6bbf => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:20:51.681231 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d65b1c8-96f3-477f-9fae-503fa29b0b99 service neutron] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:20:51.703351 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d65b1c8-96f3-477f-9fae-503fa29b0b99 service neutron] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:20:51.875580 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a92626-c3e9-4a1f-9f01-62f9efb405a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:20:51.876421 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a92626-c3e9-4a1f-9f01-62f9efb405a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:51.876689 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a92626-c3e9-4a1f-9f01-62f9efb405a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:51.878798 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a92626-c3e9-4a1f-9f01-62f9efb405a6 admin admin] PATH_INFO: `/v3/users/42cdc69f51c2497bb765c87b19314166` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:51.886014 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a92626-c3e9-4a1f-9f01-62f9efb405a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42cdc69f51c2497bb765c87b19314166)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:20:51.890331 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a92626-c3e9-4a1f-9f01-62f9efb405a6 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:20:51.983222 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:20:52.046802 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-93a92626-c3e9-4a1f-9f01-62f9efb405a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42cdc69f51c2497bb765c87b19314166'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:20:52.049407 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-93a92626-c3e9-4a1f-9f01-62f9efb405a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42cdc69f51c2497bb765c87b19314166'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:20:52.071123 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-93a92626-c3e9-4a1f-9f01-62f9efb405a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42cdc69f51c2497bb765c87b19314166'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:20:52.073809 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-93a92626-c3e9-4a1f-9f01-62f9efb405a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42cdc69f51c2497bb765c87b19314166'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:20:52.098928 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-93a92626-c3e9-4a1f-9f01-62f9efb405a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42cdc69f51c2497bb765c87b19314166'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:20:52.119250 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1243/2495] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 14:20:51 2022] DELETE /identity/v3/users/42cdc69f51c2497bb765c87b19314166 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:20:52.706203 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57784888-834a-4159-b227-97076d992b36 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:20:52.707551 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-57784888-834a-4159-b227-97076d992b36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:20:52.738998 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57784888-834a-4159-b227-97076d992b36 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:20:52.877871 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbeabc9-f4ec-48cf-aa21-99c24dbe55c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:20:52.878726 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbeabc9-f4ec-48cf-aa21-99c24dbe55c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:52.879016 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbeabc9-f4ec-48cf-aa21-99c24dbe55c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:52.879437 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbeabc9-f4ec-48cf-aa21-99c24dbe55c5 admin admin] PATH_INFO: `/v3/projects/c4ccd9e2d524414eaece4b51aaf4692d` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:52.880879 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbeabc9-f4ec-48cf-aa21-99c24dbe55c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4ccd9e2d524414eaece4b51aaf4692d)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:20:52.883943 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbeabc9-f4ec-48cf-aa21-99c24dbe55c5 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:20:52.887111 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:20:53.008118 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-bfbeabc9-f4ec-48cf-aa21-99c24dbe55c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4ccd9e2d524414eaece4b51aaf4692d', 'request_id': 'req-bfbeabc9-f4ec-48cf-aa21-99c24dbe55c5'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:20:53.037195 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-bfbeabc9-f4ec-48cf-aa21-99c24dbe55c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4ccd9e2d524414eaece4b51aaf4692d', 'request_id': 'req-bfbeabc9-f4ec-48cf-aa21-99c24dbe55c5'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:20:53.050505 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-bfbeabc9-f4ec-48cf-aa21-99c24dbe55c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4ccd9e2d524414eaece4b51aaf4692d', 'request_id': 'req-bfbeabc9-f4ec-48cf-aa21-99c24dbe55c5'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:20:53.078557 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-bfbeabc9-f4ec-48cf-aa21-99c24dbe55c5 admin admin] The token cache is being invalidate because project c4ccd9e2d524414eaece4b51aaf4692d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86933) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:20:53.078943 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-bfbeabc9-f4ec-48cf-aa21-99c24dbe55c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:20:53.080110 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-bfbeabc9-f4ec-48cf-aa21-99c24dbe55c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:20:53.082395 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1253/2496] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 14:20:52 2022] DELETE /identity/v3/projects/c4ccd9e2d524414eaece4b51aaf4692d => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:20:53.088375 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7848c08c-ac37-4cc0-be28-1b0b75f7a3ed None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:53.089110 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7848c08c-ac37-4cc0-be28-1b0b75f7a3ed None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:53.089110 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7848c08c-ac37-4cc0-be28-1b0b75f7a3ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:53.119971 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-7848c08c-ac37-4cc0-be28-1b0b75f7a3ed None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:20:53.121683 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-7848c08c-ac37-4cc0-be28-1b0b75f7a3ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:20:53.285748 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1244/2497] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:20:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:20:53.295929 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f20b40-1a01-43cc-ab70-f2d3ea2ecb1c None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:53.296567 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f20b40-1a01-43cc-ab70-f2d3ea2ecb1c None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:53.297010 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f20b40-1a01-43cc-ab70-f2d3ea2ecb1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:53.323711 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-96f20b40-1a01-43cc-ab70-f2d3ea2ecb1c None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:20:53.325552 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-96f20b40-1a01-43cc-ab70-f2d3ea2ecb1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:20:53.340542 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1254/2498] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:20:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:20:53.346212 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7848c08c-ac37-4cc0-be28-1b0b75f7a3ed None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:20:53.364894 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7848c08c-ac37-4cc0-be28-1b0b75f7a3ed None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:20:53.371532 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f75666d-9bc8-4cc8-93b3-3a1c4974257b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:20:53.372297 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f75666d-9bc8-4cc8-93b3-3a1c4974257b admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:53.373167 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f75666d-9bc8-4cc8-93b3-3a1c4974257b admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:53.373790 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f75666d-9bc8-4cc8-93b3-3a1c4974257b admin admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:53.374619 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f75666d-9bc8-4cc8-93b3-3a1c4974257b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:20:53.375150 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f75666d-9bc8-4cc8-93b3-3a1c4974257b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:20:53.378025 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f75666d-9bc8-4cc8-93b3-3a1c4974257b admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:20:53.397528 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1245/2499] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Jul 26 14:20:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:20:53.406426 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f20b40-1a01-43cc-ab70-f2d3ea2ecb1c None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:20:53.422016 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f20b40-1a01-43cc-ab70-f2d3ea2ecb1c None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:20:53.429541 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa60989-0809-4825-9b4d-f561252a00c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:20:53.439234 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa60989-0809-4825-9b4d-f561252a00c6 admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:53.439668 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa60989-0809-4825-9b4d-f561252a00c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:53.440040 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa60989-0809-4825-9b4d-f561252a00c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:53.440846 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa60989-0809-4825-9b4d-f561252a00c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:20:53.442904 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa60989-0809-4825-9b4d-f561252a00c6 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:20:53.508725 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1255/2500] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 14:20:53 2022] POST /identity/v3/projects => generated 385 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:20:53.516326 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f75666d-9bc8-4cc8-93b3-3a1c4974257b admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:20:53.535320 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f75666d-9bc8-4cc8-93b3-3a1c4974257b admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:20:53.666202 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4535437a-7c35-46f3-a56d-633a0597a087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:20:53.666974 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4535437a-7c35-46f3-a56d-633a0597a087 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:53.667333 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4535437a-7c35-46f3-a56d-633a0597a087 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:53.668012 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4535437a-7c35-46f3-a56d-633a0597a087 admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:53.669018 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4535437a-7c35-46f3-a56d-633a0597a087 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:20:53.671658 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4535437a-7c35-46f3-a56d-633a0597a087 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:20:53.715640 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1246/2501] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 14:20:53 2022] POST /identity/v3/users => generated 293 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:20:53.716951 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa60989-0809-4825-9b4d-f561252a00c6 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:20:53.731448 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa60989-0809-4825-9b4d-f561252a00c6 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:20:53.742073 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1debfd1-e826-4b4a-86c5-8ffb8a75c7fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:20:53.743150 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1debfd1-e826-4b4a-86c5-8ffb8a75c7fb admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:53.743620 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1debfd1-e826-4b4a-86c5-8ffb8a75c7fb admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:53.743899 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1debfd1-e826-4b4a-86c5-8ffb8a75c7fb admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:53.744678 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1debfd1-e826-4b4a-86c5-8ffb8a75c7fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:20:53.747914 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1debfd1-e826-4b4a-86c5-8ffb8a75c7fb admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:20:53.764408 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1256/2502] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:20:53 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:20:53.770205 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4535437a-7c35-46f3-a56d-633a0597a087 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:20:53.790379 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4535437a-7c35-46f3-a56d-633a0597a087 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:20:53.827704 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d7c0b5-1b97-4531-99f9-30d150f55fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:20:53.829181 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d7c0b5-1b97-4531-99f9-30d150f55fe6 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:53.831074 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d7c0b5-1b97-4531-99f9-30d150f55fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:53.831378 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d7c0b5-1b97-4531-99f9-30d150f55fe6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:53.832206 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d7c0b5-1b97-4531-99f9-30d150f55fe6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:20:53.835928 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d7c0b5-1b97-4531-99f9-30d150f55fe6 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:20:53.860574 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1247/2503] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:20:53 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:20:53.866628 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1debfd1-e826-4b4a-86c5-8ffb8a75c7fb admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:20:53.887821 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1debfd1-e826-4b4a-86c5-8ffb8a75c7fb admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:20:53.905279 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a07a51-4876-4834-bb73-07dc5a624173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:20:53.906520 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a07a51-4876-4834-bb73-07dc5a624173 admin admin] REQUEST_METHOD: `PUT` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:53.907088 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a07a51-4876-4834-bb73-07dc5a624173 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:53.907553 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a07a51-4876-4834-bb73-07dc5a624173 admin admin] PATH_INFO: `/v3/projects/286551312c75426bbd8d8f2072501ca1/users/a920d0a0ec09400b950f02d7d35cdfcf/roles/759f389b221049c4b01faf3b984cd8d7` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:53.942094 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a07a51-4876-4834-bb73-07dc5a624173 admin admin] RBAC: Authorizing `identity:create_grant(project_id=286551312c75426bbd8d8f2072501ca1, user_id=a920d0a0ec09400b950f02d7d35cdfcf, role_id=759f389b221049c4b01faf3b984cd8d7)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:20:53.949189 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a07a51-4876-4834-bb73-07dc5a624173 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:20:53.974707 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1257/2504] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 14:20:53 2022] PUT /identity/v3/projects/286551312c75426bbd8d8f2072501ca1/users/a920d0a0ec09400b950f02d7d35cdfcf/roles/759f389b221049c4b01faf3b984cd8d7 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:20:53.986944 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73216ac-5910-43e4-bc6f-0196f2d66bbb None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:53.987443 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73216ac-5910-43e4-bc6f-0196f2d66bbb None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:53.988043 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73216ac-5910-43e4-bc6f-0196f2d66bbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:54.020090 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:20:54.083319 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-b73216ac-5910-43e4-bc6f-0196f2d66bbb None None] MFA Rules not processed for user `a920d0a0ec09400b950f02d7d35cdfcf`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:20:54.085049 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-b73216ac-5910-43e4-bc6f-0196f2d66bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:20:54.171622 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1248/2505] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:20:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:20:54.175675 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a07a51-4876-4834-bb73-07dc5a624173 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:20:54.253355 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a07a51-4876-4834-bb73-07dc5a624173 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:20:54.259916 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d56eef-1826-40b3-90f9-f4c2354c122d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:20:54.260725 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d56eef-1826-40b3-90f9-f4c2354c122d admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:54.261109 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d56eef-1826-40b3-90f9-f4c2354c122d admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:54.261680 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d56eef-1826-40b3-90f9-f4c2354c122d admin admin] PATH_INFO: `/v3/projects` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:54.262798 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d56eef-1826-40b3-90f9-f4c2354c122d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:20:54.264802 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d56eef-1826-40b3-90f9-f4c2354c122d admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:20:54.304022 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1258/2506] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 14:20:54 2022] POST /identity/v3/projects => generated 383 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 14:20:54.305030 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73216ac-5910-43e4-bc6f-0196f2d66bbb None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:20:54.318222 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73216ac-5910-43e4-bc6f-0196f2d66bbb None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:20:54.394658 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9868485-ae87-4a07-bd56-460b903762e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:20:54.395354 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9868485-ae87-4a07-bd56-460b903762e5 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:54.395712 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9868485-ae87-4a07-bd56-460b903762e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:54.396050 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9868485-ae87-4a07-bd56-460b903762e5 admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:54.396871 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9868485-ae87-4a07-bd56-460b903762e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:20:54.399476 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9868485-ae87-4a07-bd56-460b903762e5 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:20:54.433344 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1249/2507] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 14:20:54 2022] POST /identity/v3/users => generated 298 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 14:20:54.436040 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d56eef-1826-40b3-90f9-f4c2354c122d admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:20:54.449328 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d56eef-1826-40b3-90f9-f4c2354c122d admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:20:54.456215 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d7a989-f1ad-47a5-a7ef-5eba252e36bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:20:54.459482 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d7a989-f1ad-47a5-a7ef-5eba252e36bf admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:54.460072 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d7a989-f1ad-47a5-a7ef-5eba252e36bf admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:54.460522 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d7a989-f1ad-47a5-a7ef-5eba252e36bf admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:54.461761 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d7a989-f1ad-47a5-a7ef-5eba252e36bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:20:54.463901 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d7a989-f1ad-47a5-a7ef-5eba252e36bf admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:20:54.474923 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:20:54.486644 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1259/2508] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:20:54 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:20:54.490561 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9868485-ae87-4a07-bd56-460b903762e5 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:20:54.503450 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9868485-ae87-4a07-bd56-460b903762e5 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:20:54.508879 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c431dc3-ae13-4faa-a439-4154ab598e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:20:54.509863 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c431dc3-ae13-4faa-a439-4154ab598e23 admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:54.510464 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c431dc3-ae13-4faa-a439-4154ab598e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:54.510923 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c431dc3-ae13-4faa-a439-4154ab598e23 admin admin] PATH_INFO: `/v3/projects/a82ec7e5b8794bb98660f77aee95f204/users/6466d810244c44599b95972d7e96ef2e/roles/6e0201a9a6484f7d8b112b1eafff70bd` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:54.532756 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c431dc3-ae13-4faa-a439-4154ab598e23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a82ec7e5b8794bb98660f77aee95f204, user_id=6466d810244c44599b95972d7e96ef2e, role_id=6e0201a9a6484f7d8b112b1eafff70bd)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:20:54.535643 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c431dc3-ae13-4faa-a439-4154ab598e23 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:20:54.555830 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1250/2509] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 14:20:54 2022] PUT /identity/v3/projects/a82ec7e5b8794bb98660f77aee95f204/users/6466d810244c44599b95972d7e96ef2e/roles/6e0201a9a6484f7d8b112b1eafff70bd => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:20:54.559446 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d7a989-f1ad-47a5-a7ef-5eba252e36bf admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:20:54.574053 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d7a989-f1ad-47a5-a7ef-5eba252e36bf admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:20:54.657572 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a1c6b6-9b92-4c91-8984-5d71b6a4635c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:20:54.658708 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a1c6b6-9b92-4c91-8984-5d71b6a4635c admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:54.659087 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a1c6b6-9b92-4c91-8984-5d71b6a4635c admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:54.659712 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a1c6b6-9b92-4c91-8984-5d71b6a4635c admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:54.660542 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a1c6b6-9b92-4c91-8984-5d71b6a4635c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:20:54.662743 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a1c6b6-9b92-4c91-8984-5d71b6a4635c admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:20:54.681513 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1260/2510] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:20:54 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 14:20:54.686319 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c431dc3-ae13-4faa-a439-4154ab598e23 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:20:54.703204 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c431dc3-ae13-4faa-a439-4154ab598e23 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:20:54.709304 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15d9b24-5330-4939-8cc5-ed53f8f66ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:20:54.710121 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15d9b24-5330-4939-8cc5-ed53f8f66ad0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:54.710406 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15d9b24-5330-4939-8cc5-ed53f8f66ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:54.710685 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15d9b24-5330-4939-8cc5-ed53f8f66ad0 admin admin] PATH_INFO: `/v3/projects/a82ec7e5b8794bb98660f77aee95f204/users/6466d810244c44599b95972d7e96ef2e/roles/6e0201a9a6484f7d8b112b1eafff70bd` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:54.714405 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15d9b24-5330-4939-8cc5-ed53f8f66ad0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a82ec7e5b8794bb98660f77aee95f204, user_id=6466d810244c44599b95972d7e96ef2e, role_id=6e0201a9a6484f7d8b112b1eafff70bd)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:20:54.718597 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15d9b24-5330-4939-8cc5-ed53f8f66ad0 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:20:54.746680 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1251/2511] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 14:20:54 2022] PUT /identity/v3/projects/a82ec7e5b8794bb98660f77aee95f204/users/6466d810244c44599b95972d7e96ef2e/roles/6e0201a9a6484f7d8b112b1eafff70bd => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:20:54.749866 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35849ac-3909-4f44-90f3-f7ac71f2afa8 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:54.750383 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35849ac-3909-4f44-90f3-f7ac71f2afa8 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:54.750772 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35849ac-3909-4f44-90f3-f7ac71f2afa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:54.796628 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-d35849ac-3909-4f44-90f3-f7ac71f2afa8 None None] MFA Rules not processed for user `6466d810244c44599b95972d7e96ef2e`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:20:54.799250 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-d35849ac-3909-4f44-90f3-f7ac71f2afa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:20:54.992563 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1261/2512] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:20:54 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:20:55.003455 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15d9b24-5330-4939-8cc5-ed53f8f66ad0 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:20:55.005342 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-b15d9b24-5330-4939-8cc5-ed53f8f66ad0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:20:55.176838 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:20:55.186155 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15d9b24-5330-4939-8cc5-ed53f8f66ad0 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:20:55.193008 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44f7ce9-23d4-4569-840b-5714e47365f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:20:55.195014 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44f7ce9-23d4-4569-840b-5714e47365f0 service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:55.196735 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44f7ce9-23d4-4569-840b-5714e47365f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:55.197251 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44f7ce9-23d4-4569-840b-5714e47365f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:55.214008 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35849ac-3909-4f44-90f3-f7ac71f2afa8 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:20:55.216986 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-d35849ac-3909-4f44-90f3-f7ac71f2afa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:20:55.346479 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44f7ce9-23d4-4569-840b-5714e47365f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:20:55.347947 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44f7ce9-23d4-4569-840b-5714e47365f0 service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:20:55.349051 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35849ac-3909-4f44-90f3-f7ac71f2afa8 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:20:55.356870 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b40d51f-7046-47e4-b0bf-1271a065cc9b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '835cbc13756b41b598142df3e35f5492', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:20:55.357724 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b40d51f-7046-47e4-b0bf-1271a065cc9b service placement] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:55.358133 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b40d51f-7046-47e4-b0bf-1271a065cc9b service placement] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:55.359096 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b40d51f-7046-47e4-b0bf-1271a065cc9b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:55.360610 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-1b40d51f-7046-47e4-b0bf-1271a065cc9b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:20:55.369993 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1252/2513] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:20:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:20:55.376088 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b40d51f-7046-47e4-b0bf-1271a065cc9b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:20:55.379581 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b40d51f-7046-47e4-b0bf-1271a065cc9b service placement] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:20:55.393323 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1262/2514] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:20:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:20:55.408524 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249a47a4-b895-4999-95eb-d92a8b7b67c2 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:55.408815 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249a47a4-b895-4999-95eb-d92a8b7b67c2 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:55.409083 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249a47a4-b895-4999-95eb-d92a8b7b67c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:55.460757 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-249a47a4-b895-4999-95eb-d92a8b7b67c2 None None] MFA Rules not processed for user `83d3ef25325c4fc6a0b302d459c4c617`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:20:55.462536 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-249a47a4-b895-4999-95eb-d92a8b7b67c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:20:55.582534 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1253/2515] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:20:55 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:20:55.602769 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b40d51f-7046-47e4-b0bf-1271a065cc9b service placement] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:20:55.604850 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-1b40d51f-7046-47e4-b0bf-1271a065cc9b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:20:55.628092 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:20:55.726847 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b40d51f-7046-47e4-b0bf-1271a065cc9b service placement] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:20:55.735521 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e975bb94-f0f1-4f77-9eb1-a43d6ed9f75a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c69161cd520b4b29bf50f3bf2f1c033b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:20:55.736164 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e975bb94-f0f1-4f77-9eb1-a43d6ed9f75a service nova] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:55.736475 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e975bb94-f0f1-4f77-9eb1-a43d6ed9f75a service nova] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:55.736912 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e975bb94-f0f1-4f77-9eb1-a43d6ed9f75a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:55.739111 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-e975bb94-f0f1-4f77-9eb1-a43d6ed9f75a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:20:55.816899 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e975bb94-f0f1-4f77-9eb1-a43d6ed9f75a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:20:55.818929 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e975bb94-f0f1-4f77-9eb1-a43d6ed9f75a service nova] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:20:55.831620 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1263/2516] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:20:55 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:20:58.296386 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02603516-8982-4168-a45d-3b3f6a75ef7d None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:20:58.296962 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02603516-8982-4168-a45d-3b3f6a75ef7d None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:20:58.296962 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02603516-8982-4168-a45d-3b3f6a75ef7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:20:58.316249 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-02603516-8982-4168-a45d-3b3f6a75ef7d None None] MFA Rules not processed for user `d18cab03b71a45dc812dce21798ae806`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:20:58.317456 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-02603516-8982-4168-a45d-3b3f6a75ef7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:20:58.324568 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1254/2517] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:20:58 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:21:00.011138 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e975bb94-f0f1-4f77-9eb1-a43d6ed9f75a service nova] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:21:00.030866 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e975bb94-f0f1-4f77-9eb1-a43d6ed9f75a service nova] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:21:00.056461 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac06ab2-4d97-42f0-b693-4dc9e8ed929e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:21:00.057451 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac06ab2-4d97-42f0-b693-4dc9e8ed929e service cinder] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:00.057955 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac06ab2-4d97-42f0-b693-4dc9e8ed929e service cinder] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:00.060226 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac06ab2-4d97-42f0-b693-4dc9e8ed929e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:00.070979 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac06ab2-4d97-42f0-b693-4dc9e8ed929e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:21:00.072825 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac06ab2-4d97-42f0-b693-4dc9e8ed929e service cinder] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:21:00.086614 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1264/2518] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:21:00 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:21:01.984464 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02603516-8982-4168-a45d-3b3f6a75ef7d None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:21:01.987747 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-02603516-8982-4168-a45d-3b3f6a75ef7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:21:02.020438 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02603516-8982-4168-a45d-3b3f6a75ef7d None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:21:02.070768 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac06ab2-4d97-42f0-b693-4dc9e8ed929e service cinder] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:21:02.072336 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-5ac06ab2-4d97-42f0-b693-4dc9e8ed929e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:21:02.080414 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426a843b-74aa-4b53-ace1-2d31e018b250 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '835cbc13756b41b598142df3e35f5492', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:21:02.081117 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426a843b-74aa-4b53-ace1-2d31e018b250 service placement] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:02.081508 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426a843b-74aa-4b53-ace1-2d31e018b250 service placement] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:02.081855 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426a843b-74aa-4b53-ace1-2d31e018b250 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:02.087267 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-426a843b-74aa-4b53-ace1-2d31e018b250 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:21:02.121761 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426a843b-74aa-4b53-ace1-2d31e018b250 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:21:02.125328 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426a843b-74aa-4b53-ace1-2d31e018b250 service placement] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:21:02.151140 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1255/2519] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:21:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:21:02.186944 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac06ab2-4d97-42f0-b693-4dc9e8ed929e service cinder] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:21:02.195233 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045ea4b3-23e0-4bfa-9662-1d8b69132c18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:21:02.196242 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045ea4b3-23e0-4bfa-9662-1d8b69132c18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:02.196531 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045ea4b3-23e0-4bfa-9662-1d8b69132c18 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:02.197036 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045ea4b3-23e0-4bfa-9662-1d8b69132c18 admin admin] PATH_INFO: `/v3/users/9b9e09a71d5e400081337b2b4e487ff5` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:02.198331 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045ea4b3-23e0-4bfa-9662-1d8b69132c18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b9e09a71d5e400081337b2b4e487ff5)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:21:02.200685 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045ea4b3-23e0-4bfa-9662-1d8b69132c18 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:21:02.245252 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:21:02.268949 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-045ea4b3-23e0-4bfa-9662-1d8b69132c18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b9e09a71d5e400081337b2b4e487ff5'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:21:02.283985 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-045ea4b3-23e0-4bfa-9662-1d8b69132c18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b9e09a71d5e400081337b2b4e487ff5'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:21:02.292651 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-045ea4b3-23e0-4bfa-9662-1d8b69132c18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b9e09a71d5e400081337b2b4e487ff5'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:21:02.293403 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-045ea4b3-23e0-4bfa-9662-1d8b69132c18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b9e09a71d5e400081337b2b4e487ff5'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:21:02.294779 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-045ea4b3-23e0-4bfa-9662-1d8b69132c18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b9e09a71d5e400081337b2b4e487ff5'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:21:02.315202 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1265/2520] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 14:21:02 2022] DELETE /identity/v3/users/9b9e09a71d5e400081337b2b4e487ff5 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:21:02.514452 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426a843b-74aa-4b53-ace1-2d31e018b250 service placement] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:21:02.536203 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426a843b-74aa-4b53-ace1-2d31e018b250 service placement] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:21:02.620573 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd61f23d-5b33-4c64-b02f-4c7bdef0c321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:21:02.621149 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd61f23d-5b33-4c64-b02f-4c7bdef0c321 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:02.621438 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd61f23d-5b33-4c64-b02f-4c7bdef0c321 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:02.621753 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd61f23d-5b33-4c64-b02f-4c7bdef0c321 admin admin] PATH_INFO: `/v3/projects/6fd7e897495a4482ae0f4ca2b7d44878` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:02.622938 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd61f23d-5b33-4c64-b02f-4c7bdef0c321 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6fd7e897495a4482ae0f4ca2b7d44878)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:21:02.624844 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd61f23d-5b33-4c64-b02f-4c7bdef0c321 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:21:02.636234 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:21:02.682893 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-cd61f23d-5b33-4c64-b02f-4c7bdef0c321 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6fd7e897495a4482ae0f4ca2b7d44878', 'request_id': 'req-cd61f23d-5b33-4c64-b02f-4c7bdef0c321'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:21:02.689303 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-cd61f23d-5b33-4c64-b02f-4c7bdef0c321 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6fd7e897495a4482ae0f4ca2b7d44878', 'request_id': 'req-cd61f23d-5b33-4c64-b02f-4c7bdef0c321'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:21:02.695591 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-cd61f23d-5b33-4c64-b02f-4c7bdef0c321 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6fd7e897495a4482ae0f4ca2b7d44878', 'request_id': 'req-cd61f23d-5b33-4c64-b02f-4c7bdef0c321'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:21:02.711090 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-cd61f23d-5b33-4c64-b02f-4c7bdef0c321 admin admin] The token cache is being invalidate because project 6fd7e897495a4482ae0f4ca2b7d44878 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:21:02.711384 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-cd61f23d-5b33-4c64-b02f-4c7bdef0c321 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:21:02.712239 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-cd61f23d-5b33-4c64-b02f-4c7bdef0c321 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:21:02.714028 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1256/2521] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 14:21:02 2022] DELETE /identity/v3/projects/6fd7e897495a4482ae0f4ca2b7d44878 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:21:02.718752 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045ea4b3-23e0-4bfa-9662-1d8b69132c18 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:21:02.720545 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-045ea4b3-23e0-4bfa-9662-1d8b69132c18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:21:02.733969 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045ea4b3-23e0-4bfa-9662-1d8b69132c18 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:21:02.797902 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1435169c-2ae7-4ee8-a3dd-0b59e44a8fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:21:02.798554 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1435169c-2ae7-4ee8-a3dd-0b59e44a8fa4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:02.798836 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1435169c-2ae7-4ee8-a3dd-0b59e44a8fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:02.799110 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1435169c-2ae7-4ee8-a3dd-0b59e44a8fa4 admin admin] PATH_INFO: `/v3/users/7379682915054af28fa37e0e11578d76` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:02.801034 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1435169c-2ae7-4ee8-a3dd-0b59e44a8fa4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7379682915054af28fa37e0e11578d76)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:21:02.802964 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1435169c-2ae7-4ee8-a3dd-0b59e44a8fa4 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:21:02.865277 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-1435169c-2ae7-4ee8-a3dd-0b59e44a8fa4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7379682915054af28fa37e0e11578d76'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:21:02.878107 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-1435169c-2ae7-4ee8-a3dd-0b59e44a8fa4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7379682915054af28fa37e0e11578d76'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:21:02.886356 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-1435169c-2ae7-4ee8-a3dd-0b59e44a8fa4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7379682915054af28fa37e0e11578d76'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:21:02.887208 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-1435169c-2ae7-4ee8-a3dd-0b59e44a8fa4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7379682915054af28fa37e0e11578d76'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:21:02.888671 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-1435169c-2ae7-4ee8-a3dd-0b59e44a8fa4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7379682915054af28fa37e0e11578d76'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:21:02.907369 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1266/2522] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 14:21:02 2022] DELETE /identity/v3/users/7379682915054af28fa37e0e11578d76 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:21:03.114859 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd61f23d-5b33-4c64-b02f-4c7bdef0c321 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:21:03.117185 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-cd61f23d-5b33-4c64-b02f-4c7bdef0c321 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:21:03.142717 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd61f23d-5b33-4c64-b02f-4c7bdef0c321 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:21:03.206967 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177bfae0-6002-425d-9f5c-d20bf262a2dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:21:03.207818 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177bfae0-6002-425d-9f5c-d20bf262a2dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:03.208098 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177bfae0-6002-425d-9f5c-d20bf262a2dc admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:03.208372 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177bfae0-6002-425d-9f5c-d20bf262a2dc admin admin] PATH_INFO: `/v3/projects/b629fe77bda641ec841745deab06a455` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:03.209566 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177bfae0-6002-425d-9f5c-d20bf262a2dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=b629fe77bda641ec841745deab06a455)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:21:03.211553 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177bfae0-6002-425d-9f5c-d20bf262a2dc admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:21:03.261525 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-177bfae0-6002-425d-9f5c-d20bf262a2dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b629fe77bda641ec841745deab06a455', 'request_id': 'req-177bfae0-6002-425d-9f5c-d20bf262a2dc'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:21:03.268620 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-177bfae0-6002-425d-9f5c-d20bf262a2dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b629fe77bda641ec841745deab06a455', 'request_id': 'req-177bfae0-6002-425d-9f5c-d20bf262a2dc'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:21:03.274319 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-177bfae0-6002-425d-9f5c-d20bf262a2dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b629fe77bda641ec841745deab06a455', 'request_id': 'req-177bfae0-6002-425d-9f5c-d20bf262a2dc'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:21:03.287838 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-177bfae0-6002-425d-9f5c-d20bf262a2dc admin admin] The token cache is being invalidate because project b629fe77bda641ec841745deab06a455 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:21:03.288126 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-177bfae0-6002-425d-9f5c-d20bf262a2dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:21:03.288820 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-177bfae0-6002-425d-9f5c-d20bf262a2dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:21:03.290465 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1257/2523] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 14:21:03 2022] DELETE /identity/v3/projects/b629fe77bda641ec841745deab06a455 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:21:03.295837 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd3fe69-50e2-4dec-9894-30adcea0604c None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:03.296433 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd3fe69-50e2-4dec-9894-30adcea0604c None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:03.296433 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd3fe69-50e2-4dec-9894-30adcea0604c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:03.317286 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-5cd3fe69-50e2-4dec-9894-30adcea0604c None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:21:03.318552 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-5cd3fe69-50e2-4dec-9894-30adcea0604c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:21:03.339159 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:21:03.385971 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1267/2524] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:21:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:21:03.395242 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbe08ae-6b00-4418-83cc-694e1e4408b0 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:03.395467 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbe08ae-6b00-4418-83cc-694e1e4408b0 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:03.395738 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbe08ae-6b00-4418-83cc-694e1e4408b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:03.417125 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-4dbe08ae-6b00-4418-83cc-694e1e4408b0 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:21:03.418400 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-4dbe08ae-6b00-4418-83cc-694e1e4408b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:21:03.426922 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1258/2525] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:21:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:21:03.431562 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd3fe69-50e2-4dec-9894-30adcea0604c None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:21:03.441832 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd3fe69-50e2-4dec-9894-30adcea0604c None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:21:03.446374 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e51296-1782-44ba-a68d-41a868779264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:21:03.446992 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e51296-1782-44ba-a68d-41a868779264 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:03.447348 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e51296-1782-44ba-a68d-41a868779264 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:03.447550 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e51296-1782-44ba-a68d-41a868779264 admin admin] PATH_INFO: `/v3/domains` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:03.448348 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e51296-1782-44ba-a68d-41a868779264 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:21:03.448685 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e51296-1782-44ba-a68d-41a868779264 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:21:03.450980 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e51296-1782-44ba-a68d-41a868779264 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:21:03.463579 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1268/2526] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Jul 26 14:21:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:21:03.468085 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbe08ae-6b00-4418-83cc-694e1e4408b0 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:21:03.477975 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbe08ae-6b00-4418-83cc-694e1e4408b0 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:21:03.482860 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9d5c46-dc66-45b2-a7da-ca37ddc908bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:21:03.483467 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9d5c46-dc66-45b2-a7da-ca37ddc908bc admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:03.483767 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9d5c46-dc66-45b2-a7da-ca37ddc908bc admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:03.484093 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9d5c46-dc66-45b2-a7da-ca37ddc908bc admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:03.484676 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9d5c46-dc66-45b2-a7da-ca37ddc908bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:21:03.486675 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9d5c46-dc66-45b2-a7da-ca37ddc908bc admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:21:03.510988 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:21:03.522589 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1259/2527] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 14:21:03 2022] POST /identity/v3/projects => generated 383 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:21:03.528527 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e51296-1782-44ba-a68d-41a868779264 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:21:03.538239 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e51296-1782-44ba-a68d-41a868779264 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:21:03.604159 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647e19b0-bfda-4039-9c8d-03cddacda37f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:21:03.604868 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647e19b0-bfda-4039-9c8d-03cddacda37f admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:03.605208 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647e19b0-bfda-4039-9c8d-03cddacda37f admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:03.605692 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647e19b0-bfda-4039-9c8d-03cddacda37f admin admin] PATH_INFO: `/v3/users` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:03.607304 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647e19b0-bfda-4039-9c8d-03cddacda37f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:21:03.610857 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647e19b0-bfda-4039-9c8d-03cddacda37f admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:21:03.644352 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1269/2528] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 14:21:03 2022] POST /identity/v3/users => generated 292 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 14:21:03.651120 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9d5c46-dc66-45b2-a7da-ca37ddc908bc admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:21:03.667070 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9d5c46-dc66-45b2-a7da-ca37ddc908bc admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:21:03.672335 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c88575-1c68-4a64-8e19-c46635f92a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:21:03.672954 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c88575-1c68-4a64-8e19-c46635f92a12 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:03.673234 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c88575-1c68-4a64-8e19-c46635f92a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:03.673535 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c88575-1c68-4a64-8e19-c46635f92a12 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:03.674230 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c88575-1c68-4a64-8e19-c46635f92a12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:21:03.675814 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c88575-1c68-4a64-8e19-c46635f92a12 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:21:03.687465 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1260/2529] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:21:03 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 14:21:03.692815 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647e19b0-bfda-4039-9c8d-03cddacda37f admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:21:03.705637 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647e19b0-bfda-4039-9c8d-03cddacda37f admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:21:03.710331 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9778fb58-7d3f-4505-84f7-d217c314557b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:21:03.710951 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9778fb58-7d3f-4505-84f7-d217c314557b admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:03.711233 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9778fb58-7d3f-4505-84f7-d217c314557b admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:03.711520 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9778fb58-7d3f-4505-84f7-d217c314557b admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:03.712208 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9778fb58-7d3f-4505-84f7-d217c314557b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:21:03.713836 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9778fb58-7d3f-4505-84f7-d217c314557b admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:21:03.724462 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1270/2530] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:21:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:21:03.728880 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c88575-1c68-4a64-8e19-c46635f92a12 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:21:03.740072 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c88575-1c68-4a64-8e19-c46635f92a12 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:21:03.744853 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506010c1-a5f1-4a37-9ced-230a2ecbc676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:21:03.745558 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506010c1-a5f1-4a37-9ced-230a2ecbc676 admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:03.745848 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506010c1-a5f1-4a37-9ced-230a2ecbc676 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:03.746125 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506010c1-a5f1-4a37-9ced-230a2ecbc676 admin admin] PATH_INFO: `/v3/projects/1599897c15514b80bdc9374dd5aa9f64/users/02fbfb4bcab74a3c828335ff923616a2/roles/759f389b221049c4b01faf3b984cd8d7` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:03.763523 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506010c1-a5f1-4a37-9ced-230a2ecbc676 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1599897c15514b80bdc9374dd5aa9f64, user_id=02fbfb4bcab74a3c828335ff923616a2, role_id=759f389b221049c4b01faf3b984cd8d7)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:21:03.766250 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506010c1-a5f1-4a37-9ced-230a2ecbc676 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:21:03.780093 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1261/2531] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 14:21:03 2022] PUT /identity/v3/projects/1599897c15514b80bdc9374dd5aa9f64/users/02fbfb4bcab74a3c828335ff923616a2/roles/759f389b221049c4b01faf3b984cd8d7 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:21:03.786068 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6652ffba-88a4-452c-8465-74f4bab4cfb8 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:03.786476 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6652ffba-88a4-452c-8465-74f4bab4cfb8 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:03.786811 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6652ffba-88a4-452c-8465-74f4bab4cfb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:03.807109 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-6652ffba-88a4-452c-8465-74f4bab4cfb8 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:21:03.808371 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-6652ffba-88a4-452c-8465-74f4bab4cfb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:21:03.882611 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1271/2532] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:21:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:21:03.898061 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506010c1-a5f1-4a37-9ced-230a2ecbc676 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:21:03.900291 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-506010c1-a5f1-4a37-9ced-230a2ecbc676 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:21:03.959515 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506010c1-a5f1-4a37-9ced-230a2ecbc676 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:21:03.964498 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d07837-4e90-44b2-9e79-1721968f3034 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:21:03.965306 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d07837-4e90-44b2-9e79-1721968f3034 service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:03.966446 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d07837-4e90-44b2-9e79-1721968f3034 service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:03.966763 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d07837-4e90-44b2-9e79-1721968f3034 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:03.977046 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d07837-4e90-44b2-9e79-1721968f3034 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:21:03.978640 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d07837-4e90-44b2-9e79-1721968f3034 service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:21:03.989127 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1262/2533] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:21:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 14:21:05.286787 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6652ffba-88a4-452c-8465-74f4bab4cfb8 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:21:05.288503 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-6652ffba-88a4-452c-8465-74f4bab4cfb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:21:05.301755 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6652ffba-88a4-452c-8465-74f4bab4cfb8 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:21:05.306566 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da2e0fa-f336-48fd-92a3-fde08b5ff3fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:21:05.307357 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da2e0fa-f336-48fd-92a3-fde08b5ff3fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:05.307743 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da2e0fa-f336-48fd-92a3-fde08b5ff3fe admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:05.308261 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da2e0fa-f336-48fd-92a3-fde08b5ff3fe admin admin] PATH_INFO: `/v3/users/a920d0a0ec09400b950f02d7d35cdfcf` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:05.310245 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da2e0fa-f336-48fd-92a3-fde08b5ff3fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=a920d0a0ec09400b950f02d7d35cdfcf)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:21:05.312304 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da2e0fa-f336-48fd-92a3-fde08b5ff3fe admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:21:05.339230 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:21:05.377322 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-8da2e0fa-f336-48fd-92a3-fde08b5ff3fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a920d0a0ec09400b950f02d7d35cdfcf'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:21:05.387963 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-8da2e0fa-f336-48fd-92a3-fde08b5ff3fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a920d0a0ec09400b950f02d7d35cdfcf'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:21:05.401189 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-8da2e0fa-f336-48fd-92a3-fde08b5ff3fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a920d0a0ec09400b950f02d7d35cdfcf'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:21:05.402321 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-8da2e0fa-f336-48fd-92a3-fde08b5ff3fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a920d0a0ec09400b950f02d7d35cdfcf'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:21:05.403726 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-8da2e0fa-f336-48fd-92a3-fde08b5ff3fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a920d0a0ec09400b950f02d7d35cdfcf'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:21:05.445140 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1272/2534] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 14:21:05 2022] DELETE /identity/v3/users/a920d0a0ec09400b950f02d7d35cdfcf => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:21:05.451821 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db53938-ddcb-40c5-9fcb-fb65f3d96c0e None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:05.452649 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db53938-ddcb-40c5-9fcb-fb65f3d96c0e None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:05.453047 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db53938-ddcb-40c5-9fcb-fb65f3d96c0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:05.490691 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-0db53938-ddcb-40c5-9fcb-fb65f3d96c0e None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:21:05.491983 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-0db53938-ddcb-40c5-9fcb-fb65f3d96c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:21:05.598644 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1263/2535] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:21:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:21:05.621053 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da2e0fa-f336-48fd-92a3-fde08b5ff3fe admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:21:05.624337 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-8da2e0fa-f336-48fd-92a3-fde08b5ff3fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:21:05.761686 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da2e0fa-f336-48fd-92a3-fde08b5ff3fe admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:21:05.782944 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7289a7-aaba-413f-bd28-d6668fd3b288 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:21:05.784647 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7289a7-aaba-413f-bd28-d6668fd3b288 service neutron] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:05.785702 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7289a7-aaba-413f-bd28-d6668fd3b288 service neutron] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:05.786452 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7289a7-aaba-413f-bd28-d6668fd3b288 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:05.824187 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7289a7-aaba-413f-bd28-d6668fd3b288 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:21:05.826922 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7289a7-aaba-413f-bd28-d6668fd3b288 service neutron] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:21:05.848560 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1273/2536] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:21:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:21:06.259556 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db53938-ddcb-40c5-9fcb-fb65f3d96c0e None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:21:06.262180 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-0db53938-ddcb-40c5-9fcb-fb65f3d96c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:21:06.310754 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db53938-ddcb-40c5-9fcb-fb65f3d96c0e None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:21:06.329874 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520f846c-3c1a-44e6-9a2b-5bd6e8fde4ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:21:06.331500 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520f846c-3c1a-44e6-9a2b-5bd6e8fde4ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:06.333256 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520f846c-3c1a-44e6-9a2b-5bd6e8fde4ef admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:06.334596 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520f846c-3c1a-44e6-9a2b-5bd6e8fde4ef admin admin] PATH_INFO: `/v3/projects/286551312c75426bbd8d8f2072501ca1` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:06.340146 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520f846c-3c1a-44e6-9a2b-5bd6e8fde4ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=286551312c75426bbd8d8f2072501ca1)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:21:06.344223 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520f846c-3c1a-44e6-9a2b-5bd6e8fde4ef admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:21:06.465548 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-520f846c-3c1a-44e6-9a2b-5bd6e8fde4ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '286551312c75426bbd8d8f2072501ca1', 'request_id': 'req-520f846c-3c1a-44e6-9a2b-5bd6e8fde4ef'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:21:06.483715 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-520f846c-3c1a-44e6-9a2b-5bd6e8fde4ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '286551312c75426bbd8d8f2072501ca1', 'request_id': 'req-520f846c-3c1a-44e6-9a2b-5bd6e8fde4ef'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:21:06.497740 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-520f846c-3c1a-44e6-9a2b-5bd6e8fde4ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '286551312c75426bbd8d8f2072501ca1', 'request_id': 'req-520f846c-3c1a-44e6-9a2b-5bd6e8fde4ef'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:21:06.502343 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:21:06.528022 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-520f846c-3c1a-44e6-9a2b-5bd6e8fde4ef admin admin] The token cache is being invalidate because project 286551312c75426bbd8d8f2072501ca1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:21:06.528851 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-520f846c-3c1a-44e6-9a2b-5bd6e8fde4ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:21:06.529923 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-520f846c-3c1a-44e6-9a2b-5bd6e8fde4ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:21:06.534951 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1264/2537] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 14:21:06 2022] DELETE /identity/v3/projects/286551312c75426bbd8d8f2072501ca1 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:21:06.536378 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7289a7-aaba-413f-bd28-d6668fd3b288 service neutron] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:21:06.552805 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7289a7-aaba-413f-bd28-d6668fd3b288 service neutron] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:21:06.635649 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8df442-23f5-4c2b-af29-2af40347b4ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:21:06.636543 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8df442-23f5-4c2b-af29-2af40347b4ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:06.636953 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8df442-23f5-4c2b-af29-2af40347b4ad admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:06.637339 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8df442-23f5-4c2b-af29-2af40347b4ad admin admin] PATH_INFO: `/v3/users/6466d810244c44599b95972d7e96ef2e` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:06.643064 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8df442-23f5-4c2b-af29-2af40347b4ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=6466d810244c44599b95972d7e96ef2e)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:21:06.645330 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8df442-23f5-4c2b-af29-2af40347b4ad admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:21:06.720482 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:21:06.732451 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-6f8df442-23f5-4c2b-af29-2af40347b4ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6466d810244c44599b95972d7e96ef2e'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:21:06.743548 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-6f8df442-23f5-4c2b-af29-2af40347b4ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6466d810244c44599b95972d7e96ef2e'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:21:06.760867 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-6f8df442-23f5-4c2b-af29-2af40347b4ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6466d810244c44599b95972d7e96ef2e'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:21:06.761765 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-6f8df442-23f5-4c2b-af29-2af40347b4ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6466d810244c44599b95972d7e96ef2e'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:21:06.762402 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-6f8df442-23f5-4c2b-af29-2af40347b4ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6466d810244c44599b95972d7e96ef2e'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:21:06.794270 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1274/2538] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 14:21:06 2022] DELETE /identity/v3/users/6466d810244c44599b95972d7e96ef2e => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:21:07.193123 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520f846c-3c1a-44e6-9a2b-5bd6e8fde4ef admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:21:07.194625 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-520f846c-3c1a-44e6-9a2b-5bd6e8fde4ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:21:07.218095 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520f846c-3c1a-44e6-9a2b-5bd6e8fde4ef admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:21:07.315209 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4436d51-23d9-40c0-8aa4-d3523020fd04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:21:07.317203 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4436d51-23d9-40c0-8aa4-d3523020fd04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:07.318369 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4436d51-23d9-40c0-8aa4-d3523020fd04 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:07.318867 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4436d51-23d9-40c0-8aa4-d3523020fd04 admin admin] PATH_INFO: `/v3/projects/a82ec7e5b8794bb98660f77aee95f204` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:07.321214 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4436d51-23d9-40c0-8aa4-d3523020fd04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a82ec7e5b8794bb98660f77aee95f204)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:21:07.324120 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4436d51-23d9-40c0-8aa4-d3523020fd04 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:21:07.439487 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-f4436d51-23d9-40c0-8aa4-d3523020fd04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a82ec7e5b8794bb98660f77aee95f204', 'request_id': 'req-f4436d51-23d9-40c0-8aa4-d3523020fd04'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:21:07.457793 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-f4436d51-23d9-40c0-8aa4-d3523020fd04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a82ec7e5b8794bb98660f77aee95f204', 'request_id': 'req-f4436d51-23d9-40c0-8aa4-d3523020fd04'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:21:07.468256 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-f4436d51-23d9-40c0-8aa4-d3523020fd04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a82ec7e5b8794bb98660f77aee95f204', 'request_id': 'req-f4436d51-23d9-40c0-8aa4-d3523020fd04'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:21:07.502733 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-f4436d51-23d9-40c0-8aa4-d3523020fd04 admin admin] The token cache is being invalidate because project a82ec7e5b8794bb98660f77aee95f204 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:21:07.503403 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-f4436d51-23d9-40c0-8aa4-d3523020fd04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:21:07.504651 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-f4436d51-23d9-40c0-8aa4-d3523020fd04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:21:07.506464 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1265/2539] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 14:21:07 2022] DELETE /identity/v3/projects/a82ec7e5b8794bb98660f77aee95f204 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:21:07.514724 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98601aa6-ec55-4a6a-8d00-9e91b94f4f0e None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:07.515056 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98601aa6-ec55-4a6a-8d00-9e91b94f4f0e None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:07.515336 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98601aa6-ec55-4a6a-8d00-9e91b94f4f0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:07.567576 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-98601aa6-ec55-4a6a-8d00-9e91b94f4f0e None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:21:07.569736 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-98601aa6-ec55-4a6a-8d00-9e91b94f4f0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:21:07.701658 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1275/2540] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:21:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:21:07.710595 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e96b818-5c03-45b4-b84b-7d070f08bc4e None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:07.710976 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e96b818-5c03-45b4-b84b-7d070f08bc4e None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:07.711261 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e96b818-5c03-45b4-b84b-7d070f08bc4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:07.739875 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-1e96b818-5c03-45b4-b84b-7d070f08bc4e None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:21:07.740771 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-1e96b818-5c03-45b4-b84b-7d070f08bc4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:21:07.756087 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1266/2541] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:21:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:21:07.759312 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98601aa6-ec55-4a6a-8d00-9e91b94f4f0e None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:21:07.771307 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98601aa6-ec55-4a6a-8d00-9e91b94f4f0e None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:21:07.779232 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7fd71f-014f-4199-b227-aa9858fefc34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:21:07.780504 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7fd71f-014f-4199-b227-aa9858fefc34 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:07.780885 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7fd71f-014f-4199-b227-aa9858fefc34 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:07.781335 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7fd71f-014f-4199-b227-aa9858fefc34 admin admin] PATH_INFO: `/v3/domains` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:07.782274 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7fd71f-014f-4199-b227-aa9858fefc34 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:21:07.782681 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7fd71f-014f-4199-b227-aa9858fefc34 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:21:07.785263 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7fd71f-014f-4199-b227-aa9858fefc34 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:21:07.800444 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1276/2542] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Jul 26 14:21:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:21:07.810531 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e96b818-5c03-45b4-b84b-7d070f08bc4e None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:21:07.822272 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e96b818-5c03-45b4-b84b-7d070f08bc4e None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:21:07.827552 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9485206-c872-411a-927b-4861fe448f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:21:07.828415 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9485206-c872-411a-927b-4861fe448f54 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:07.828854 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9485206-c872-411a-927b-4861fe448f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:07.829303 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9485206-c872-411a-927b-4861fe448f54 admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:07.830035 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9485206-c872-411a-927b-4861fe448f54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:21:07.831977 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9485206-c872-411a-927b-4861fe448f54 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:21:07.875503 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7fd71f-014f-4199-b227-aa9858fefc34 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:21:07.877807 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1267/2543] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 14:21:07 2022] POST /identity/v3/projects => generated 371 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:21:07.888904 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7fd71f-014f-4199-b227-aa9858fefc34 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:21:07.990398 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e194fee3-aa09-4a06-a24e-0e5bf80ebc87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:21:07.990992 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e194fee3-aa09-4a06-a24e-0e5bf80ebc87 admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:07.991386 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e194fee3-aa09-4a06-a24e-0e5bf80ebc87 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:07.991767 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e194fee3-aa09-4a06-a24e-0e5bf80ebc87 admin admin] PATH_INFO: `/v3/users` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:07.992715 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e194fee3-aa09-4a06-a24e-0e5bf80ebc87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:21:07.995094 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e194fee3-aa09-4a06-a24e-0e5bf80ebc87 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:21:08.071419 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1277/2544] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 14:21:07 2022] POST /identity/v3/users => generated 286 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:21:08.074406 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9485206-c872-411a-927b-4861fe448f54 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:21:08.087781 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9485206-c872-411a-927b-4861fe448f54 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:21:08.106112 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596c63bb-8484-483f-aa7e-b0079fbea193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:21:08.106958 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596c63bb-8484-483f-aa7e-b0079fbea193 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:08.107249 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596c63bb-8484-483f-aa7e-b0079fbea193 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:08.107519 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596c63bb-8484-483f-aa7e-b0079fbea193 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:08.108246 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596c63bb-8484-483f-aa7e-b0079fbea193 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:21:08.109957 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596c63bb-8484-483f-aa7e-b0079fbea193 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:21:08.123373 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1268/2545] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:21:08 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 14:21:08.127751 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e194fee3-aa09-4a06-a24e-0e5bf80ebc87 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:21:08.144851 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e194fee3-aa09-4a06-a24e-0e5bf80ebc87 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:21:08.172546 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5eebd0-dd9b-4826-9c7b-9ecf39508a46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:21:08.174316 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5eebd0-dd9b-4826-9c7b-9ecf39508a46 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:08.174596 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5eebd0-dd9b-4826-9c7b-9ecf39508a46 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:08.174782 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5eebd0-dd9b-4826-9c7b-9ecf39508a46 admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:08.176614 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5eebd0-dd9b-4826-9c7b-9ecf39508a46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:21:08.178328 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5eebd0-dd9b-4826-9c7b-9ecf39508a46 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:21:08.206956 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1278/2546] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:21:08 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:21:08.211407 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596c63bb-8484-483f-aa7e-b0079fbea193 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:21:08.224674 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596c63bb-8484-483f-aa7e-b0079fbea193 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:21:08.237771 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0af60c-5f5a-4b38-9ceb-b3f8174bd748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:21:08.238551 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0af60c-5f5a-4b38-9ceb-b3f8174bd748 admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:08.238881 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0af60c-5f5a-4b38-9ceb-b3f8174bd748 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:08.239299 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0af60c-5f5a-4b38-9ceb-b3f8174bd748 admin admin] PATH_INFO: `/v3/projects/76aa2f48491048e797e6eedd2ead4e51/users/df978a63cdd54d1aaba1a986457d0be9/roles/759f389b221049c4b01faf3b984cd8d7` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:08.259717 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0af60c-5f5a-4b38-9ceb-b3f8174bd748 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76aa2f48491048e797e6eedd2ead4e51, user_id=df978a63cdd54d1aaba1a986457d0be9, role_id=759f389b221049c4b01faf3b984cd8d7)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:21:08.262525 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0af60c-5f5a-4b38-9ceb-b3f8174bd748 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:21:08.287852 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1269/2547] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 14:21:08 2022] PUT /identity/v3/projects/76aa2f48491048e797e6eedd2ead4e51/users/df978a63cdd54d1aaba1a986457d0be9/roles/759f389b221049c4b01faf3b984cd8d7 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:21:08.304051 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c61c3-7930-480a-b002-d6c891eed41d None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:08.304342 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c61c3-7930-480a-b002-d6c891eed41d None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:08.304623 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c61c3-7930-480a-b002-d6c891eed41d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:08.343206 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-088c61c3-7930-480a-b002-d6c891eed41d None None] MFA Rules not processed for user `df978a63cdd54d1aaba1a986457d0be9`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:21:08.344873 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-088c61c3-7930-480a-b002-d6c891eed41d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:21:08.364232 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:21:08.438242 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1279/2548] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:21:08 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:21:08.454914 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0af60c-5f5a-4b38-9ceb-b3f8174bd748 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:21:08.456834 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-8f0af60c-5f5a-4b38-9ceb-b3f8174bd748 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:21:08.483258 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:INSERT=4,keystone:DELETE=8,keystone:SELECT=88 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:21:08.542748 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0af60c-5f5a-4b38-9ceb-b3f8174bd748 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:21:08.548127 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf1e983-fb7a-432a-a96b-29d441f5a274 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:21:08.549525 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf1e983-fb7a-432a-a96b-29d441f5a274 service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:08.549859 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf1e983-fb7a-432a-a96b-29d441f5a274 service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:08.550155 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf1e983-fb7a-432a-a96b-29d441f5a274 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:08.566111 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf1e983-fb7a-432a-a96b-29d441f5a274 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:21:08.568008 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf1e983-fb7a-432a-a96b-29d441f5a274 service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:21:08.580682 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1270/2549] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:21:08 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:21:08.607201 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088c61c3-7930-480a-b002-d6c891eed41d None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:21:08.695993 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088c61c3-7930-480a-b002-d6c891eed41d None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:21:08.701566 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5c0d4c-42d6-4320-8fde-cd6e520e5e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:21:08.702251 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5c0d4c-42d6-4320-8fde-cd6e520e5e0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:08.702530 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5c0d4c-42d6-4320-8fde-cd6e520e5e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:08.703268 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5c0d4c-42d6-4320-8fde-cd6e520e5e0d admin admin] PATH_INFO: `/v3/users/df978a63cdd54d1aaba1a986457d0be9` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:08.703995 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5c0d4c-42d6-4320-8fde-cd6e520e5e0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=df978a63cdd54d1aaba1a986457d0be9)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:21:08.705990 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5c0d4c-42d6-4320-8fde-cd6e520e5e0d admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:21:08.774176 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-1f5c0d4c-42d6-4320-8fde-cd6e520e5e0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df978a63cdd54d1aaba1a986457d0be9'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:21:08.785875 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-1f5c0d4c-42d6-4320-8fde-cd6e520e5e0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df978a63cdd54d1aaba1a986457d0be9'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:21:08.795935 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-1f5c0d4c-42d6-4320-8fde-cd6e520e5e0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df978a63cdd54d1aaba1a986457d0be9'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:21:08.797973 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-1f5c0d4c-42d6-4320-8fde-cd6e520e5e0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df978a63cdd54d1aaba1a986457d0be9'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:21:08.798705 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-1f5c0d4c-42d6-4320-8fde-cd6e520e5e0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df978a63cdd54d1aaba1a986457d0be9'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:21:08.830286 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f934e3-659b-487f-8020-e8b4419f868b None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:08.830557 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f934e3-659b-487f-8020-e8b4419f868b None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:08.830878 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f934e3-659b-487f-8020-e8b4419f868b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:08.833176 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1280/2550] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 14:21:08 2022] DELETE /identity/v3/users/df978a63cdd54d1aaba1a986457d0be9 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:21:08.853372 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-67f934e3-659b-487f-8020-e8b4419f868b None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:21:08.854174 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-67f934e3-659b-487f-8020-e8b4419f868b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:21:08.984806 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1271/2551] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:21:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:21:08.994058 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5c0d4c-42d6-4320-8fde-cd6e520e5e0d admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:21:08.996810 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-1f5c0d4c-42d6-4320-8fde-cd6e520e5e0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:21:09.068219 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:21:09.094356 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5c0d4c-42d6-4320-8fde-cd6e520e5e0d admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:21:09.103833 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce5801a-bade-4100-b7bd-0260081ef536 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:09.104233 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce5801a-bade-4100-b7bd-0260081ef536 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:09.104574 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce5801a-bade-4100-b7bd-0260081ef536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:09.108146 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467df772-15b8-462e-ae96-d85e44fedba7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:21:09.109025 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467df772-15b8-462e-ae96-d85e44fedba7 service neutron] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:09.109443 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467df772-15b8-462e-ae96-d85e44fedba7 service neutron] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:09.109994 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467df772-15b8-462e-ae96-d85e44fedba7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:09.136135 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467df772-15b8-462e-ae96-d85e44fedba7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:21:09.138721 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467df772-15b8-462e-ae96-d85e44fedba7 service neutron] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:21:09.157011 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1281/2552] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:21:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:21:09.158374 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-cce5801a-bade-4100-b7bd-0260081ef536 None None] MFA Rules not processed for user `02fbfb4bcab74a3c828335ff923616a2`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:21:09.162094 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-cce5801a-bade-4100-b7bd-0260081ef536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:21:09.295568 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1272/2553] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:21:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:21:09.300096 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467df772-15b8-462e-ae96-d85e44fedba7 service neutron] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:21:09.303541 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-467df772-15b8-462e-ae96-d85e44fedba7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:21:09.321691 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467df772-15b8-462e-ae96-d85e44fedba7 service neutron] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:21:09.327501 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd80c647-4d68-4c9b-91bc-f25779427be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:21:09.328264 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd80c647-4d68-4c9b-91bc-f25779427be4 admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:09.328550 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd80c647-4d68-4c9b-91bc-f25779427be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:09.328826 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd80c647-4d68-4c9b-91bc-f25779427be4 admin admin] PATH_INFO: `/v3/projects` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:09.329618 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd80c647-4d68-4c9b-91bc-f25779427be4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:21:09.331841 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd80c647-4d68-4c9b-91bc-f25779427be4 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:21:09.380798 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1282/2554] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 14:21:09 2022] POST /identity/v3/projects => generated 385 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:21:09.384905 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce5801a-bade-4100-b7bd-0260081ef536 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:21:09.406945 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce5801a-bade-4100-b7bd-0260081ef536 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:21:09.488293 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:21:09.510529 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266ae618-2116-440f-a42f-3a9ce72c2bb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:21:09.511295 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266ae618-2116-440f-a42f-3a9ce72c2bb6 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:09.511650 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266ae618-2116-440f-a42f-3a9ce72c2bb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:09.512029 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266ae618-2116-440f-a42f-3a9ce72c2bb6 admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:09.513878 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266ae618-2116-440f-a42f-3a9ce72c2bb6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:21:09.521324 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266ae618-2116-440f-a42f-3a9ce72c2bb6 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:21:09.547720 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1273/2555] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 14:21:09 2022] POST /identity/v3/users => generated 299 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:21:09.552061 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd80c647-4d68-4c9b-91bc-f25779427be4 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:21:09.570480 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd80c647-4d68-4c9b-91bc-f25779427be4 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:21:09.577724 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e585791-5cfb-4148-97f1-5ec032189d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:21:09.578501 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e585791-5cfb-4148-97f1-5ec032189d42 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:09.578803 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e585791-5cfb-4148-97f1-5ec032189d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:09.579238 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e585791-5cfb-4148-97f1-5ec032189d42 admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:09.580383 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e585791-5cfb-4148-97f1-5ec032189d42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:21:09.582427 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e585791-5cfb-4148-97f1-5ec032189d42 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:21:09.598582 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1283/2556] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:21:09 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:21:09.605693 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266ae618-2116-440f-a42f-3a9ce72c2bb6 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:21:09.610546 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e585791-5cfb-4148-97f1-5ec032189d42 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:21:09.613449 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-1e585791-5cfb-4148-97f1-5ec032189d42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:21:09.623043 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266ae618-2116-440f-a42f-3a9ce72c2bb6 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:21:09.635435 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2faedc7-0828-4242-b393-fb89f9040d60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:21:09.636017 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2faedc7-0828-4242-b393-fb89f9040d60 admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:09.636307 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2faedc7-0828-4242-b393-fb89f9040d60 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:09.636592 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2faedc7-0828-4242-b393-fb89f9040d60 admin admin] PATH_INFO: `/v3/projects/891db561dc1b424c900eae1fd95f5afb/users/fe25209f8e2249d69d0ed28552826519/roles/6e0201a9a6484f7d8b112b1eafff70bd` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:09.645651 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e585791-5cfb-4148-97f1-5ec032189d42 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:21:09.651024 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d24cae-006a-4b3a-9692-c4b5f6f19baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:21:09.651800 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d24cae-006a-4b3a-9692-c4b5f6f19baa admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:09.652149 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d24cae-006a-4b3a-9692-c4b5f6f19baa admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:09.661790 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d24cae-006a-4b3a-9692-c4b5f6f19baa admin admin] PATH_INFO: `/v3/projects/76aa2f48491048e797e6eedd2ead4e51` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:09.663518 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2faedc7-0828-4242-b393-fb89f9040d60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=891db561dc1b424c900eae1fd95f5afb, user_id=fe25209f8e2249d69d0ed28552826519, role_id=6e0201a9a6484f7d8b112b1eafff70bd)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:21:09.666349 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2faedc7-0828-4242-b393-fb89f9040d60 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:21:09.670208 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d24cae-006a-4b3a-9692-c4b5f6f19baa admin admin] RBAC: Authorizing `identity:delete_project(project_id=76aa2f48491048e797e6eedd2ead4e51)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:21:09.673665 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d24cae-006a-4b3a-9692-c4b5f6f19baa admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:21:09.685414 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1274/2557] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 14:21:09 2022] PUT /identity/v3/projects/891db561dc1b424c900eae1fd95f5afb/users/fe25209f8e2249d69d0ed28552826519/roles/6e0201a9a6484f7d8b112b1eafff70bd => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:21:09.689450 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2faedc7-0828-4242-b393-fb89f9040d60 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:21:09.744176 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-58d24cae-006a-4b3a-9692-c4b5f6f19baa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76aa2f48491048e797e6eedd2ead4e51', 'request_id': 'req-58d24cae-006a-4b3a-9692-c4b5f6f19baa'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:21:09.761814 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-58d24cae-006a-4b3a-9692-c4b5f6f19baa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76aa2f48491048e797e6eedd2ead4e51', 'request_id': 'req-58d24cae-006a-4b3a-9692-c4b5f6f19baa'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:21:09.770711 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-58d24cae-006a-4b3a-9692-c4b5f6f19baa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76aa2f48491048e797e6eedd2ead4e51', 'request_id': 'req-58d24cae-006a-4b3a-9692-c4b5f6f19baa'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:21:09.772077 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2faedc7-0828-4242-b393-fb89f9040d60 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:21:09.786574 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-58d24cae-006a-4b3a-9692-c4b5f6f19baa admin admin] The token cache is being invalidate because project 76aa2f48491048e797e6eedd2ead4e51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86933) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:21:09.786906 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-58d24cae-006a-4b3a-9692-c4b5f6f19baa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:21:09.787938 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-58d24cae-006a-4b3a-9692-c4b5f6f19baa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:21:09.789681 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1284/2558] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 14:21:09 2022] DELETE /identity/v3/projects/76aa2f48491048e797e6eedd2ead4e51 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:21:09.796258 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f009f5a8-3a7b-4c40-88ec-635364ade69d None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:09.796871 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f009f5a8-3a7b-4c40-88ec-635364ade69d None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:09.797256 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f009f5a8-3a7b-4c40-88ec-635364ade69d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:09.820844 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-f009f5a8-3a7b-4c40-88ec-635364ade69d None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:21:09.822481 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-f009f5a8-3a7b-4c40-88ec-635364ade69d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:21:09.850944 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9f5f80-40a4-4b77-875d-239960c65d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:21:09.851850 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9f5f80-40a4-4b77-875d-239960c65d3a admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:09.853381 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9f5f80-40a4-4b77-875d-239960c65d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:09.853381 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9f5f80-40a4-4b77-875d-239960c65d3a admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:09.853381 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9f5f80-40a4-4b77-875d-239960c65d3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:21:09.855726 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9f5f80-40a4-4b77-875d-239960c65d3a admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:21:09.886990 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1275/2559] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:21:09 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 14:21:09.888313 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9f5f80-40a4-4b77-875d-239960c65d3a admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:21:09.895199 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-ea9f5f80-40a4-4b77-875d-239960c65d3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:21:09.912911 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9f5f80-40a4-4b77-875d-239960c65d3a admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:21:09.918123 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4769e01-06e8-4a18-86f2-7640a571bbe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:21:09.918835 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4769e01-06e8-4a18-86f2-7640a571bbe9 admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:09.919124 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4769e01-06e8-4a18-86f2-7640a571bbe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:09.919445 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4769e01-06e8-4a18-86f2-7640a571bbe9 admin admin] PATH_INFO: `/v3/projects/891db561dc1b424c900eae1fd95f5afb/users/fe25209f8e2249d69d0ed28552826519/roles/6e0201a9a6484f7d8b112b1eafff70bd` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:09.921558 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4769e01-06e8-4a18-86f2-7640a571bbe9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=891db561dc1b424c900eae1fd95f5afb, user_id=fe25209f8e2249d69d0ed28552826519, role_id=6e0201a9a6484f7d8b112b1eafff70bd)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:21:09.924248 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4769e01-06e8-4a18-86f2-7640a571bbe9 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:21:09.930882 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1285/2560] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:21:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:21:09.940207 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1276/2561] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 14:21:09 2022] PUT /identity/v3/projects/891db561dc1b424c900eae1fd95f5afb/users/fe25209f8e2249d69d0ed28552826519/roles/6e0201a9a6484f7d8b112b1eafff70bd => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:21:09.942602 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a504fb3-1b58-489b-b161-3312ba0786e6 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:09.942753 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a504fb3-1b58-489b-b161-3312ba0786e6 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:09.943359 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a504fb3-1b58-489b-b161-3312ba0786e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:09.967489 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-9a504fb3-1b58-489b-b161-3312ba0786e6 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:21:09.968754 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-9a504fb3-1b58-489b-b161-3312ba0786e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:21:10.008292 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:21:10.084635 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1286/2562] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:21:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:21:10.088798 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4769e01-06e8-4a18-86f2-7640a571bbe9 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:21:10.105734 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4769e01-06e8-4a18-86f2-7640a571bbe9 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:21:10.123627 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01de748b-b5f3-4c8b-bc7d-44eb61129644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:21:10.124024 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01de748b-b5f3-4c8b-bc7d-44eb61129644 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:10.124318 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01de748b-b5f3-4c8b-bc7d-44eb61129644 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:10.124596 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01de748b-b5f3-4c8b-bc7d-44eb61129644 admin admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:10.125531 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01de748b-b5f3-4c8b-bc7d-44eb61129644 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:21:10.125828 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01de748b-b5f3-4c8b-bc7d-44eb61129644 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:21:10.128316 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01de748b-b5f3-4c8b-bc7d-44eb61129644 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:21:10.147727 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1277/2563] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Jul 26 14:21:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:21:10.152684 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a504fb3-1b58-489b-b161-3312ba0786e6 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:21:10.164544 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a504fb3-1b58-489b-b161-3312ba0786e6 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:21:10.173186 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309ff527-2a45-4980-ab3c-4b15921df6c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:21:10.173781 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309ff527-2a45-4980-ab3c-4b15921df6c9 admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:10.174069 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309ff527-2a45-4980-ab3c-4b15921df6c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:10.174347 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309ff527-2a45-4980-ab3c-4b15921df6c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:10.175032 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309ff527-2a45-4980-ab3c-4b15921df6c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:21:10.177105 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309ff527-2a45-4980-ab3c-4b15921df6c9 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:21:10.245913 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1287/2564] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 14:21:10 2022] POST /identity/v3/projects => generated 383 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:21:10.250522 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01de748b-b5f3-4c8b-bc7d-44eb61129644 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:21:10.264572 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01de748b-b5f3-4c8b-bc7d-44eb61129644 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:21:10.342352 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:21:10.381131 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1309261-8ecb-4163-bb88-1bb5d5e14666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:21:10.382278 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1309261-8ecb-4163-bb88-1bb5d5e14666 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:10.382505 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1309261-8ecb-4163-bb88-1bb5d5e14666 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:10.383404 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1309261-8ecb-4163-bb88-1bb5d5e14666 admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:10.385119 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1309261-8ecb-4163-bb88-1bb5d5e14666 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:21:10.388302 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1309261-8ecb-4163-bb88-1bb5d5e14666 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:21:10.463827 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1278/2565] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 14:21:10 2022] POST /identity/v3/users => generated 292 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:21:10.465730 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309ff527-2a45-4980-ab3c-4b15921df6c9 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:21:10.476844 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309ff527-2a45-4980-ab3c-4b15921df6c9 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:21:10.489372 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787a61a0-f0d8-4ca4-96b7-575a5db3f97a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:21:10.490082 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787a61a0-f0d8-4ca4-96b7-575a5db3f97a admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:10.490543 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787a61a0-f0d8-4ca4-96b7-575a5db3f97a admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:10.490830 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787a61a0-f0d8-4ca4-96b7-575a5db3f97a admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:10.491551 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787a61a0-f0d8-4ca4-96b7-575a5db3f97a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:21:10.493451 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787a61a0-f0d8-4ca4-96b7-575a5db3f97a admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:21:10.529698 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1288/2566] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:21:10 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 14:21:10.531535 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1309261-8ecb-4163-bb88-1bb5d5e14666 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:21:10.543457 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1309261-8ecb-4163-bb88-1bb5d5e14666 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:21:10.548936 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0801d180-50ff-4137-b878-4c7cf8e882e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:21:10.549613 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0801d180-50ff-4137-b878-4c7cf8e882e7 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:10.549899 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0801d180-50ff-4137-b878-4c7cf8e882e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:10.550174 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0801d180-50ff-4137-b878-4c7cf8e882e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:10.551006 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0801d180-50ff-4137-b878-4c7cf8e882e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:21:10.552630 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0801d180-50ff-4137-b878-4c7cf8e882e7 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:21:10.567897 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1279/2567] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:21:10 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 14:21:10.573206 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787a61a0-f0d8-4ca4-96b7-575a5db3f97a admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:21:10.587369 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787a61a0-f0d8-4ca4-96b7-575a5db3f97a admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:21:10.593637 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cc8031-3962-4242-b3bd-6df9890604fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:21:10.594585 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cc8031-3962-4242-b3bd-6df9890604fe admin admin] REQUEST_METHOD: `PUT` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:10.594712 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cc8031-3962-4242-b3bd-6df9890604fe admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:10.594988 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cc8031-3962-4242-b3bd-6df9890604fe admin admin] PATH_INFO: `/v3/projects/6f02ad8864b5495cafa76444f9ed63d8/users/61b7edf024f64b3499f756c1a09b01b1/roles/759f389b221049c4b01faf3b984cd8d7` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:10.612929 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cc8031-3962-4242-b3bd-6df9890604fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f02ad8864b5495cafa76444f9ed63d8, user_id=61b7edf024f64b3499f756c1a09b01b1, role_id=759f389b221049c4b01faf3b984cd8d7)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:21:10.615839 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cc8031-3962-4242-b3bd-6df9890604fe admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:21:10.632440 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1289/2568] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 14:21:10 2022] PUT /identity/v3/projects/6f02ad8864b5495cafa76444f9ed63d8/users/61b7edf024f64b3499f756c1a09b01b1/roles/759f389b221049c4b01faf3b984cd8d7 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:21:10.644201 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0011e8fa-639c-4581-b186-3111aa0b8f5e None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:10.644543 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0011e8fa-639c-4581-b186-3111aa0b8f5e None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:10.644910 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0011e8fa-639c-4581-b186-3111aa0b8f5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:10.691057 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-0011e8fa-639c-4581-b186-3111aa0b8f5e None None] MFA Rules not processed for user `61b7edf024f64b3499f756c1a09b01b1`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:21:10.692684 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-0011e8fa-639c-4581-b186-3111aa0b8f5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:21:10.823957 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1280/2569] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:21:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:21:10.843965 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cc8031-3962-4242-b3bd-6df9890604fe admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:21:10.846306 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-b0cc8031-3962-4242-b3bd-6df9890604fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:21:10.929763 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cc8031-3962-4242-b3bd-6df9890604fe admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:21:10.935284 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ec8896-af82-4403-8cad-7ab4a8f4d62c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:21:10.936250 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ec8896-af82-4403-8cad-7ab4a8f4d62c service cinder] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:10.936661 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ec8896-af82-4403-8cad-7ab4a8f4d62c service cinder] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:10.936882 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ec8896-af82-4403-8cad-7ab4a8f4d62c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:10.946785 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ec8896-af82-4403-8cad-7ab4a8f4d62c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:21:10.949515 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ec8896-af82-4403-8cad-7ab4a8f4d62c service cinder] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:21:10.959051 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1290/2570] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:21:10 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:21:13.835056 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0011e8fa-639c-4581-b186-3111aa0b8f5e None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:21:14.030847 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0011e8fa-639c-4581-b186-3111aa0b8f5e None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:21:14.044581 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc54a3b-a236-406a-b090-8dc68273735f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:21:14.045416 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc54a3b-a236-406a-b090-8dc68273735f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:14.047295 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc54a3b-a236-406a-b090-8dc68273735f admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:14.047702 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc54a3b-a236-406a-b090-8dc68273735f admin admin] PATH_INFO: `/v3/users/61b7edf024f64b3499f756c1a09b01b1` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:14.053161 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc54a3b-a236-406a-b090-8dc68273735f admin admin] RBAC: Authorizing `identity:delete_user(user_id=61b7edf024f64b3499f756c1a09b01b1)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:21:14.066202 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc54a3b-a236-406a-b090-8dc68273735f admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:21:14.206591 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:21:14.213600 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-4fc54a3b-a236-406a-b090-8dc68273735f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61b7edf024f64b3499f756c1a09b01b1'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:21:14.219628 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-4fc54a3b-a236-406a-b090-8dc68273735f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61b7edf024f64b3499f756c1a09b01b1'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:21:14.228453 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-4fc54a3b-a236-406a-b090-8dc68273735f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61b7edf024f64b3499f756c1a09b01b1'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:21:14.229262 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-4fc54a3b-a236-406a-b090-8dc68273735f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61b7edf024f64b3499f756c1a09b01b1'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:21:14.248398 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-4fc54a3b-a236-406a-b090-8dc68273735f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61b7edf024f64b3499f756c1a09b01b1'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:21:14.269515 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1281/2571] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 14:21:13 2022] DELETE /identity/v3/users/61b7edf024f64b3499f756c1a09b01b1 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:21:14.275153 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35586341-7828-4d26-85c9-d2f7750a4c29 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:14.276480 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35586341-7828-4d26-85c9-d2f7750a4c29 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:14.277162 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35586341-7828-4d26-85c9-d2f7750a4c29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:14.310361 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-35586341-7828-4d26-85c9-d2f7750a4c29 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:21:14.311215 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-35586341-7828-4d26-85c9-d2f7750a4c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:21:14.407100 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1291/2572] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:21:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:21:14.408369 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc54a3b-a236-406a-b090-8dc68273735f admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:21:14.409949 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-4fc54a3b-a236-406a-b090-8dc68273735f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:21:14.520460 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc54a3b-a236-406a-b090-8dc68273735f admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:21:14.528385 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1c6a76-19c2-4841-ba1c-537412343428 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:21:14.529679 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c6a76-19c2-4841-ba1c-537412343428 service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:14.530160 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c6a76-19c2-4841-ba1c-537412343428 service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:14.531423 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c6a76-19c2-4841-ba1c-537412343428 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:14.547592 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1c6a76-19c2-4841-ba1c-537412343428 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:21:14.551034 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1c6a76-19c2-4841-ba1c-537412343428 service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:21:14.582363 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1282/2573] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:21:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:21:15.100327 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35586341-7828-4d26-85c9-d2f7750a4c29 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:21:15.102390 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-35586341-7828-4d26-85c9-d2f7750a4c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:21:15.134886 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35586341-7828-4d26-85c9-d2f7750a4c29 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:21:15.144098 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b861fe8-4463-48dd-9394-f428efb4a3af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:21:15.144773 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b861fe8-4463-48dd-9394-f428efb4a3af admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:15.145050 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b861fe8-4463-48dd-9394-f428efb4a3af admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:15.145324 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b861fe8-4463-48dd-9394-f428efb4a3af admin admin] PATH_INFO: `/v3/projects/6f02ad8864b5495cafa76444f9ed63d8` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:15.148883 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b861fe8-4463-48dd-9394-f428efb4a3af admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f02ad8864b5495cafa76444f9ed63d8)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:21:15.151916 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b861fe8-4463-48dd-9394-f428efb4a3af admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:21:15.172293 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:21:15.272653 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-6b861fe8-4463-48dd-9394-f428efb4a3af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f02ad8864b5495cafa76444f9ed63d8', 'request_id': 'req-6b861fe8-4463-48dd-9394-f428efb4a3af'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:21:15.286196 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-6b861fe8-4463-48dd-9394-f428efb4a3af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f02ad8864b5495cafa76444f9ed63d8', 'request_id': 'req-6b861fe8-4463-48dd-9394-f428efb4a3af'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:21:15.302751 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-6b861fe8-4463-48dd-9394-f428efb4a3af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f02ad8864b5495cafa76444f9ed63d8', 'request_id': 'req-6b861fe8-4463-48dd-9394-f428efb4a3af'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:21:15.323288 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-6b861fe8-4463-48dd-9394-f428efb4a3af admin admin] The token cache is being invalidate because project 6f02ad8864b5495cafa76444f9ed63d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86933) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:21:15.323713 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-6b861fe8-4463-48dd-9394-f428efb4a3af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:21:15.324364 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-6b861fe8-4463-48dd-9394-f428efb4a3af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:21:15.326348 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1292/2574] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 14:21:15 2022] DELETE /identity/v3/projects/6f02ad8864b5495cafa76444f9ed63d8 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:21:15.332741 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c0e919-91e1-428e-bc78-cb7aa93e2a8c None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:15.333529 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c0e919-91e1-428e-bc78-cb7aa93e2a8c None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:15.334035 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c0e919-91e1-428e-bc78-cb7aa93e2a8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:15.362389 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-17c0e919-91e1-428e-bc78-cb7aa93e2a8c None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:21:15.363769 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-17c0e919-91e1-428e-bc78-cb7aa93e2a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:21:15.436232 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1283/2575] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:21:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:21:15.446979 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c74da28-4991-4578-9434-39e2c416e2fc None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:15.447526 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c74da28-4991-4578-9434-39e2c416e2fc None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:15.448544 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c74da28-4991-4578-9434-39e2c416e2fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:15.476605 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-5c74da28-4991-4578-9434-39e2c416e2fc None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:21:15.478643 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-5c74da28-4991-4578-9434-39e2c416e2fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:21:15.492511 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1293/2576] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:21:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:21:15.493737 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c0e919-91e1-428e-bc78-cb7aa93e2a8c None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:21:15.506368 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c0e919-91e1-428e-bc78-cb7aa93e2a8c None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:21:15.514825 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e357596e-b838-49de-89d0-98d2af31a416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:21:15.515812 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e357596e-b838-49de-89d0-98d2af31a416 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:15.516298 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e357596e-b838-49de-89d0-98d2af31a416 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:15.517183 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e357596e-b838-49de-89d0-98d2af31a416 admin admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:15.518620 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e357596e-b838-49de-89d0-98d2af31a416 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:21:15.519121 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e357596e-b838-49de-89d0-98d2af31a416 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:21:15.521146 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e357596e-b838-49de-89d0-98d2af31a416 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:21:15.538282 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1284/2577] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Jul 26 14:21:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:21:15.543509 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c74da28-4991-4578-9434-39e2c416e2fc None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:21:15.557336 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c74da28-4991-4578-9434-39e2c416e2fc None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:21:15.564680 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5edc1e-9739-4237-9165-603030cc072e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:21:15.566054 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5edc1e-9739-4237-9165-603030cc072e admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:15.566547 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5edc1e-9739-4237-9165-603030cc072e admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:15.567344 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5edc1e-9739-4237-9165-603030cc072e admin admin] PATH_INFO: `/v3/projects` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:15.568411 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5edc1e-9739-4237-9165-603030cc072e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:21:15.571768 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5edc1e-9739-4237-9165-603030cc072e admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:21:15.614912 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1294/2578] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 14:21:15 2022] POST /identity/v3/projects => generated 371 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:21:15.619023 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e357596e-b838-49de-89d0-98d2af31a416 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:21:15.630497 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e357596e-b838-49de-89d0-98d2af31a416 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:21:15.694651 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:21:15.713346 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379e0d2f-a42d-494e-afc3-dda0a1ac6eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:21:15.714709 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379e0d2f-a42d-494e-afc3-dda0a1ac6eaf admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:15.715655 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379e0d2f-a42d-494e-afc3-dda0a1ac6eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:15.715655 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379e0d2f-a42d-494e-afc3-dda0a1ac6eaf admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:15.716910 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379e0d2f-a42d-494e-afc3-dda0a1ac6eaf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:21:15.719594 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379e0d2f-a42d-494e-afc3-dda0a1ac6eaf admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:21:15.754876 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1285/2579] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 14:21:15 2022] POST /identity/v3/users => generated 286 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:21:15.761788 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5edc1e-9739-4237-9165-603030cc072e admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:21:15.780092 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5edc1e-9739-4237-9165-603030cc072e admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:21:15.794453 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6386e211-c4ba-4bba-89ac-8e285d3b3ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:21:15.795483 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6386e211-c4ba-4bba-89ac-8e285d3b3ad2 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:15.795951 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6386e211-c4ba-4bba-89ac-8e285d3b3ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:15.796881 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6386e211-c4ba-4bba-89ac-8e285d3b3ad2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:15.797840 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6386e211-c4ba-4bba-89ac-8e285d3b3ad2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:21:15.800631 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6386e211-c4ba-4bba-89ac-8e285d3b3ad2 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:21:15.814607 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1295/2580] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:21:15 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 14:21:15.818301 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379e0d2f-a42d-494e-afc3-dda0a1ac6eaf admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:21:15.839915 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379e0d2f-a42d-494e-afc3-dda0a1ac6eaf admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:21:15.849295 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979bdc55-ac3d-4b6b-a968-a2384f889aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:21:15.849989 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979bdc55-ac3d-4b6b-a968-a2384f889aae admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:15.850365 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979bdc55-ac3d-4b6b-a968-a2384f889aae admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:15.850551 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979bdc55-ac3d-4b6b-a968-a2384f889aae admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:15.851252 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979bdc55-ac3d-4b6b-a968-a2384f889aae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:21:15.853052 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979bdc55-ac3d-4b6b-a968-a2384f889aae admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:21:15.874038 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1286/2581] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:21:15 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 14:21:15.876004 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6386e211-c4ba-4bba-89ac-8e285d3b3ad2 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:21:15.893523 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6386e211-c4ba-4bba-89ac-8e285d3b3ad2 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:21:15.908596 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea48845-3517-41f2-90fd-37814d148dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:21:15.910726 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea48845-3517-41f2-90fd-37814d148dc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:15.911030 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea48845-3517-41f2-90fd-37814d148dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:15.911317 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea48845-3517-41f2-90fd-37814d148dc1 admin admin] PATH_INFO: `/v3/projects/f8902384644743399cb8efb017becf9b/users/6bee9c24298049d0825ef991b1ecc21f/roles/759f389b221049c4b01faf3b984cd8d7` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:15.947374 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37137687-9281-4b76-8e62-4ad5513a86f1 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:15.947711 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37137687-9281-4b76-8e62-4ad5513a86f1 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:15.948768 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37137687-9281-4b76-8e62-4ad5513a86f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:15.949288 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea48845-3517-41f2-90fd-37814d148dc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8902384644743399cb8efb017becf9b, user_id=6bee9c24298049d0825ef991b1ecc21f, role_id=759f389b221049c4b01faf3b984cd8d7)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:21:15.953116 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea48845-3517-41f2-90fd-37814d148dc1 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:21:15.982003 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1296/2582] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 14:21:15 2022] PUT /identity/v3/projects/f8902384644743399cb8efb017becf9b/users/6bee9c24298049d0825ef991b1ecc21f/roles/759f389b221049c4b01faf3b984cd8d7 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:21:15.993676 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a9e137-a0fe-401c-a50e-3f3d8cc5c000 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:15.994626 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a9e137-a0fe-401c-a50e-3f3d8cc5c000 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:15.995254 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a9e137-a0fe-401c-a50e-3f3d8cc5c000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:16.008046 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-37137687-9281-4b76-8e62-4ad5513a86f1 None None] MFA Rules not processed for user `fe25209f8e2249d69d0ed28552826519`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:21:16.009401 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-37137687-9281-4b76-8e62-4ad5513a86f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:21:16.062069 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-40a9e137-a0fe-401c-a50e-3f3d8cc5c000 None None] MFA Rules not processed for user `6bee9c24298049d0825ef991b1ecc21f`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:21:16.063660 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-40a9e137-a0fe-401c-a50e-3f3d8cc5c000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:21:16.156199 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1287/2583] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:21:15 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:21:16.171972 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37137687-9281-4b76-8e62-4ad5513a86f1 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:21:16.172663 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1297/2584] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:21:15 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:21:16.178704 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-37137687-9281-4b76-8e62-4ad5513a86f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:21:16.185182 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a9e137-a0fe-401c-a50e-3f3d8cc5c000 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:21:16.186676 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-40a9e137-a0fe-401c-a50e-3f3d8cc5c000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:21:16.221941 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:21:16.293303 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37137687-9281-4b76-8e62-4ad5513a86f1 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:21:16.294361 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a9e137-a0fe-401c-a50e-3f3d8cc5c000 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:21:16.299050 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e55490b-86a9-4465-b6ca-edf5c00d209f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf065aef9a046b2adaec34816a336b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:21:16.299774 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e55490b-86a9-4465-b6ca-edf5c00d209f service glance] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:16.300109 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e55490b-86a9-4465-b6ca-edf5c00d209f service glance] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:16.300401 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e55490b-86a9-4465-b6ca-edf5c00d209f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:16.301300 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56e3419-db1a-4759-b4b1-f970d438595e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf065aef9a046b2adaec34816a336b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:21:16.302330 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56e3419-db1a-4759-b4b1-f970d438595e service glance] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:16.302548 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-4e55490b-86a9-4465-b6ca-edf5c00d209f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:21:16.302896 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56e3419-db1a-4759-b4b1-f970d438595e service glance] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:16.303566 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56e3419-db1a-4759-b4b1-f970d438595e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:16.320215 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56e3419-db1a-4759-b4b1-f970d438595e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:21:16.324090 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56e3419-db1a-4759-b4b1-f970d438595e service glance] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:21:16.343714 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1298/2585] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:21:16 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 14:21:16.371440 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:21:16.389575 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e55490b-86a9-4465-b6ca-edf5c00d209f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:21:16.392282 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e55490b-86a9-4465-b6ca-edf5c00d209f service glance] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:21:16.406151 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1288/2586] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:21:16 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 14:21:16.689828 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ca0ad1-7ebc-4116-8638-333983b4b46a None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:16.690118 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ca0ad1-7ebc-4116-8638-333983b4b46a None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:16.690399 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ca0ad1-7ebc-4116-8638-333983b4b46a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:16.777841 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-91ca0ad1-7ebc-4116-8638-333983b4b46a None None] MFA Rules not processed for user `b64bd7b183d24d8c9037b261c7884810`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:21:16.779122 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-91ca0ad1-7ebc-4116-8638-333983b4b46a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:21:16.938366 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1299/2587] 10.222.0.58 () {58 vars in 1009 bytes} [Tue Jul 26 14:21:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:21:17.042304 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e55490b-86a9-4465-b6ca-edf5c00d209f service glance] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:21:17.044558 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-4e55490b-86a9-4465-b6ca-edf5c00d209f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:21:17.271981 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e55490b-86a9-4465-b6ca-edf5c00d209f service glance] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:21:17.278707 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c75aa4-5009-40cb-942d-22d9f58bae92 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab2819bac844163ae4305e4f2186605', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:21:17.279498 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c75aa4-5009-40cb-942d-22d9f58bae92 service swift] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:17.279787 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c75aa4-5009-40cb-942d-22d9f58bae92 service swift] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:17.280063 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c75aa4-5009-40cb-942d-22d9f58bae92 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:17.292945 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c75aa4-5009-40cb-942d-22d9f58bae92 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:21:17.295339 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c75aa4-5009-40cb-942d-22d9f58bae92 service swift] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:21:17.306558 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1289/2588] 10.222.0.58 () {60 vars in 1384 bytes} [Tue Jul 26 14:21:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 14:21:21.051723 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ca0ad1-7ebc-4116-8638-333983b4b46a None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:21:21.055886 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-91ca0ad1-7ebc-4116-8638-333983b4b46a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:21:21.174657 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ca0ad1-7ebc-4116-8638-333983b4b46a None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:21:21.179568 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f1b437-ce6a-4ef2-9f0c-40612708f571 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:21:21.180249 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f1b437-ce6a-4ef2-9f0c-40612708f571 service neutron] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:21.180606 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f1b437-ce6a-4ef2-9f0c-40612708f571 service neutron] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:21.180959 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f1b437-ce6a-4ef2-9f0c-40612708f571 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:21.183097 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-64f1b437-ce6a-4ef2-9f0c-40612708f571 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:21:21.196805 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f1b437-ce6a-4ef2-9f0c-40612708f571 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:21:21.199205 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f1b437-ce6a-4ef2-9f0c-40612708f571 service neutron] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:21:21.214425 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1300/2589] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:21:21 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:21:27.302479 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=46 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:21:30.307188 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ea4649-346d-4a6c-a378-2b49c3849a8a None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:30.308174 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ea4649-346d-4a6c-a378-2b49c3849a8a None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:30.308174 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ea4649-346d-4a6c-a378-2b49c3849a8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:30.333303 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-86ea4649-346d-4a6c-a378-2b49c3849a8a None None] MFA Rules not processed for user `02fbfb4bcab74a3c828335ff923616a2`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:21:30.334816 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-86ea4649-346d-4a6c-a378-2b49c3849a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:21:30.344481 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1290/2590] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:21:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:21:31.207637 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=89 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:21:32.794430 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf5cdeb-e905-43e5-98c2-2059f1940867 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:32.795549 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf5cdeb-e905-43e5-98c2-2059f1940867 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:32.795549 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf5cdeb-e905-43e5-98c2-2059f1940867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:32.818794 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-ddf5cdeb-e905-43e5-98c2-2059f1940867 None None] MFA Rules not processed for user `b64bd7b183d24d8c9037b261c7884810`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:21:32.820229 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-ddf5cdeb-e905-43e5-98c2-2059f1940867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:21:32.836318 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1301/2591] 10.222.0.58 () {58 vars in 1009 bytes} [Tue Jul 26 14:21:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:21:32.856724 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ea4649-346d-4a6c-a378-2b49c3849a8a None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:21:32.876836 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ea4649-346d-4a6c-a378-2b49c3849a8a None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:21:32.884510 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521a5c42-e1f9-44a5-b1ca-5b9981456e00 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab2819bac844163ae4305e4f2186605', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:21:32.885449 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521a5c42-e1f9-44a5-b1ca-5b9981456e00 service swift] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:32.885923 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521a5c42-e1f9-44a5-b1ca-5b9981456e00 service swift] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:32.886345 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521a5c42-e1f9-44a5-b1ca-5b9981456e00 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:32.903230 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521a5c42-e1f9-44a5-b1ca-5b9981456e00 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:21:32.905790 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521a5c42-e1f9-44a5-b1ca-5b9981456e00 service swift] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:21:32.932659 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1291/2592] 10.222.0.58 () {60 vars in 1384 bytes} [Tue Jul 26 14:21:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 14:21:38.336076 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf5cdeb-e905-43e5-98c2-2059f1940867 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:21:38.350442 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf5cdeb-e905-43e5-98c2-2059f1940867 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:21:38.354564 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e33970-7e51-48ac-a701-6221bee57aee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:21:38.358214 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e33970-7e51-48ac-a701-6221bee57aee service neutron] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:38.358214 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e33970-7e51-48ac-a701-6221bee57aee service neutron] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:38.358214 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e33970-7e51-48ac-a701-6221bee57aee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:38.358636 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-25e33970-7e51-48ac-a701-6221bee57aee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:21:38.483147 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e33970-7e51-48ac-a701-6221bee57aee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:21:38.484703 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e33970-7e51-48ac-a701-6221bee57aee service neutron] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:21:38.510389 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1302/2593] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:21:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 14:21:42.922057 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=9 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:21:43.798703 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521a5c42-e1f9-44a5-b1ca-5b9981456e00 service swift] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:21:43.800377 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-521a5c42-e1f9-44a5-b1ca-5b9981456e00 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:21:43.869835 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521a5c42-e1f9-44a5-b1ca-5b9981456e00 service swift] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:21:43.874172 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e5183a-f0e0-487a-bb83-b2908984cd37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:21:43.874804 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e5183a-f0e0-487a-bb83-b2908984cd37 service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:43.875085 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e5183a-f0e0-487a-bb83-b2908984cd37 service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:43.875361 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e5183a-f0e0-487a-bb83-b2908984cd37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:43.877159 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-41e5183a-f0e0-487a-bb83-b2908984cd37 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:21:43.971544 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e5183a-f0e0-487a-bb83-b2908984cd37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:21:43.973039 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e5183a-f0e0-487a-bb83-b2908984cd37 service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:21:44.005624 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1292/2594] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:21:43 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:21:46.878634 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e33970-7e51-48ac-a701-6221bee57aee service neutron] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:21:46.880436 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-25e33970-7e51-48ac-a701-6221bee57aee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:21:46.895354 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e33970-7e51-48ac-a701-6221bee57aee service neutron] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:21:46.899322 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bceceb-ee74-452a-933d-0e48b761e62b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:21:46.899953 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bceceb-ee74-452a-933d-0e48b761e62b service neutron] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:46.900242 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bceceb-ee74-452a-933d-0e48b761e62b service neutron] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:46.900609 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bceceb-ee74-452a-933d-0e48b761e62b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:46.903081 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-38bceceb-ee74-452a-933d-0e48b761e62b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:21:47.022352 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bceceb-ee74-452a-933d-0e48b761e62b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:21:47.024835 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bceceb-ee74-452a-933d-0e48b761e62b service neutron] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:21:47.044349 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1303/2595] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:21:46 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:21:49.650769 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e5183a-f0e0-487a-bb83-b2908984cd37 service cinder] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:21:49.655878 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-41e5183a-f0e0-487a-bb83-b2908984cd37 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:21:49.701138 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e5183a-f0e0-487a-bb83-b2908984cd37 service cinder] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:21:49.730525 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bd732f-cad7-46af-adb0-c6e8df035efd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c69161cd520b4b29bf50f3bf2f1c033b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:21:49.730525 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bd732f-cad7-46af-adb0-c6e8df035efd service nova] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:49.730525 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bd732f-cad7-46af-adb0-c6e8df035efd service nova] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:49.730525 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bd732f-cad7-46af-adb0-c6e8df035efd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:49.731239 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-b2bd732f-cad7-46af-adb0-c6e8df035efd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:21:49.743095 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bd732f-cad7-46af-adb0-c6e8df035efd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:21:49.743095 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bd732f-cad7-46af-adb0-c6e8df035efd service nova] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:21:49.752645 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1293/2596] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:21:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:21:50.785574 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bceceb-ee74-452a-933d-0e48b761e62b service neutron] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:21:50.872742 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bceceb-ee74-452a-933d-0e48b761e62b service neutron] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:21:50.878458 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77a3bcc-ca40-4f59-a21b-a583a042062f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:21:50.879213 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77a3bcc-ca40-4f59-a21b-a583a042062f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:50.879614 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77a3bcc-ca40-4f59-a21b-a583a042062f admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:50.880004 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77a3bcc-ca40-4f59-a21b-a583a042062f admin admin] PATH_INFO: `/v3/users/6bee9c24298049d0825ef991b1ecc21f` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:50.881461 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77a3bcc-ca40-4f59-a21b-a583a042062f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6bee9c24298049d0825ef991b1ecc21f)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:21:50.883821 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77a3bcc-ca40-4f59-a21b-a583a042062f admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:21:50.907181 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:21:50.961946 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-e77a3bcc-ca40-4f59-a21b-a583a042062f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6bee9c24298049d0825ef991b1ecc21f'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:21:50.976616 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-e77a3bcc-ca40-4f59-a21b-a583a042062f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6bee9c24298049d0825ef991b1ecc21f'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:21:50.993467 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-e77a3bcc-ca40-4f59-a21b-a583a042062f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6bee9c24298049d0825ef991b1ecc21f'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:21:50.994318 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-e77a3bcc-ca40-4f59-a21b-a583a042062f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6bee9c24298049d0825ef991b1ecc21f'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:21:50.994988 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-e77a3bcc-ca40-4f59-a21b-a583a042062f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6bee9c24298049d0825ef991b1ecc21f'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:21:51.023617 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1304/2597] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 14:21:50 2022] DELETE /identity/v3/users/6bee9c24298049d0825ef991b1ecc21f => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:21:51.031051 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3a873d-40c5-47d4-b924-b57d1618d34f None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:51.033076 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3a873d-40c5-47d4-b924-b57d1618d34f None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:51.033907 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3a873d-40c5-47d4-b924-b57d1618d34f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:51.062545 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-2a3a873d-40c5-47d4-b924-b57d1618d34f None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:21:51.064647 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-2a3a873d-40c5-47d4-b924-b57d1618d34f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:21:51.163826 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1294/2598] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:21:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:21:51.177083 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77a3bcc-ca40-4f59-a21b-a583a042062f admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:21:51.179481 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-e77a3bcc-ca40-4f59-a21b-a583a042062f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:21:51.311450 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77a3bcc-ca40-4f59-a21b-a583a042062f admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:21:51.319272 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b47d79-5952-4bb3-b3a4-813da782dc93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:21:51.320430 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b47d79-5952-4bb3-b3a4-813da782dc93 service neutron] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:51.320920 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b47d79-5952-4bb3-b3a4-813da782dc93 service neutron] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:51.321816 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b47d79-5952-4bb3-b3a4-813da782dc93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:51.340584 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b47d79-5952-4bb3-b3a4-813da782dc93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:21:51.343395 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b47d79-5952-4bb3-b3a4-813da782dc93 service neutron] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:21:51.379592 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1305/2599] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:21:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 14:21:51.894681 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3a873d-40c5-47d4-b924-b57d1618d34f None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:21:51.902178 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-2a3a873d-40c5-47d4-b924-b57d1618d34f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:21:51.946344 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3a873d-40c5-47d4-b924-b57d1618d34f None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:21:51.958239 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16143a2-8ffe-458c-97a5-9c4d48c21f82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:21:51.959352 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16143a2-8ffe-458c-97a5-9c4d48c21f82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:51.959841 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16143a2-8ffe-458c-97a5-9c4d48c21f82 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:51.960335 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16143a2-8ffe-458c-97a5-9c4d48c21f82 admin admin] PATH_INFO: `/v3/projects/f8902384644743399cb8efb017becf9b` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:51.962024 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16143a2-8ffe-458c-97a5-9c4d48c21f82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8902384644743399cb8efb017becf9b)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:21:51.964067 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16143a2-8ffe-458c-97a5-9c4d48c21f82 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:21:52.014353 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:21:52.057179 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-a16143a2-8ffe-458c-97a5-9c4d48c21f82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8902384644743399cb8efb017becf9b', 'request_id': 'req-a16143a2-8ffe-458c-97a5-9c4d48c21f82'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:21:52.069771 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-a16143a2-8ffe-458c-97a5-9c4d48c21f82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8902384644743399cb8efb017becf9b', 'request_id': 'req-a16143a2-8ffe-458c-97a5-9c4d48c21f82'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:21:52.081345 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-a16143a2-8ffe-458c-97a5-9c4d48c21f82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8902384644743399cb8efb017becf9b', 'request_id': 'req-a16143a2-8ffe-458c-97a5-9c4d48c21f82'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:21:52.102912 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-a16143a2-8ffe-458c-97a5-9c4d48c21f82 admin admin] The token cache is being invalidate because project f8902384644743399cb8efb017becf9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:21:52.103428 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-a16143a2-8ffe-458c-97a5-9c4d48c21f82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:21:52.104564 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-a16143a2-8ffe-458c-97a5-9c4d48c21f82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:21:52.107423 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1295/2600] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 14:21:51 2022] DELETE /identity/v3/projects/f8902384644743399cb8efb017becf9b => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:21:52.115020 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57f7dd7-ef63-4370-9dc4-15f8f0fcffae None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:52.115811 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57f7dd7-ef63-4370-9dc4-15f8f0fcffae None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:52.115811 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57f7dd7-ef63-4370-9dc4-15f8f0fcffae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:52.140295 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-f57f7dd7-ef63-4370-9dc4-15f8f0fcffae None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:21:52.142346 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-f57f7dd7-ef63-4370-9dc4-15f8f0fcffae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:21:52.246635 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1306/2601] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:21:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:21:52.252629 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8051f9-ee95-434d-952c-ed00e1f1084f None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:52.252989 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8051f9-ee95-434d-952c-ed00e1f1084f None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:52.253322 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8051f9-ee95-434d-952c-ed00e1f1084f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:52.279074 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-6d8051f9-ee95-434d-952c-ed00e1f1084f None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:21:52.280389 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-6d8051f9-ee95-434d-952c-ed00e1f1084f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:21:52.288773 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1296/2602] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:21:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:21:52.294573 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57f7dd7-ef63-4370-9dc4-15f8f0fcffae None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:21:52.313371 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57f7dd7-ef63-4370-9dc4-15f8f0fcffae None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:21:52.322738 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e6f9b3-863d-4a1f-ad71-2ebe3b9f8d68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:21:52.323852 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e6f9b3-863d-4a1f-ad71-2ebe3b9f8d68 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:52.324336 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e6f9b3-863d-4a1f-ad71-2ebe3b9f8d68 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:52.324814 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e6f9b3-863d-4a1f-ad71-2ebe3b9f8d68 admin admin] PATH_INFO: `/v3/domains` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:52.326145 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e6f9b3-863d-4a1f-ad71-2ebe3b9f8d68 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:21:52.326694 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e6f9b3-863d-4a1f-ad71-2ebe3b9f8d68 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:21:52.329455 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e6f9b3-863d-4a1f-ad71-2ebe3b9f8d68 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:21:52.350019 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1307/2603] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Jul 26 14:21:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:21:52.356997 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8051f9-ee95-434d-952c-ed00e1f1084f None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:21:52.371502 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8051f9-ee95-434d-952c-ed00e1f1084f None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:21:52.382174 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e01aabf-c226-4beb-ae95-961a1dd424b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:21:52.383287 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e01aabf-c226-4beb-ae95-961a1dd424b9 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:52.383790 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e01aabf-c226-4beb-ae95-961a1dd424b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:52.384281 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e01aabf-c226-4beb-ae95-961a1dd424b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:52.385589 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e01aabf-c226-4beb-ae95-961a1dd424b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:21:52.389438 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e01aabf-c226-4beb-ae95-961a1dd424b9 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:21:52.438585 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1297/2604] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 14:21:52 2022] POST /identity/v3/projects => generated 375 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:21:52.444478 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e6f9b3-863d-4a1f-ad71-2ebe3b9f8d68 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:21:52.462137 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e6f9b3-863d-4a1f-ad71-2ebe3b9f8d68 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:21:52.494978 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:21:52.564658 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aa4f04-39ab-4a27-bd56-7e6ff22fc55f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:21:52.565632 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa4f04-39ab-4a27-bd56-7e6ff22fc55f admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:52.566065 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa4f04-39ab-4a27-bd56-7e6ff22fc55f admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:52.566489 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa4f04-39ab-4a27-bd56-7e6ff22fc55f admin admin] PATH_INFO: `/v3/users` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:52.567517 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aa4f04-39ab-4a27-bd56-7e6ff22fc55f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:21:52.570718 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aa4f04-39ab-4a27-bd56-7e6ff22fc55f admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:21:52.605208 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1308/2605] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 14:21:52 2022] POST /identity/v3/users => generated 288 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:21:52.611950 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e01aabf-c226-4beb-ae95-961a1dd424b9 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:21:52.629947 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e01aabf-c226-4beb-ae95-961a1dd424b9 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:21:52.648353 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f6fd94-fcfe-4fbe-958a-77596a546766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:21:52.649453 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f6fd94-fcfe-4fbe-958a-77596a546766 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:52.650073 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f6fd94-fcfe-4fbe-958a-77596a546766 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:52.650559 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f6fd94-fcfe-4fbe-958a-77596a546766 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:52.652416 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f6fd94-fcfe-4fbe-958a-77596a546766 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:21:52.655507 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f6fd94-fcfe-4fbe-958a-77596a546766 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:21:52.675154 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1298/2606] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:21:52 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:21:52.681370 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aa4f04-39ab-4a27-bd56-7e6ff22fc55f admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:21:52.700704 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aa4f04-39ab-4a27-bd56-7e6ff22fc55f admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:21:52.708477 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e00d00-7fb8-4487-a1a9-4408c37e67ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:21:52.709544 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e00d00-7fb8-4487-a1a9-4408c37e67ba admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:52.710047 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e00d00-7fb8-4487-a1a9-4408c37e67ba admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:52.710535 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e00d00-7fb8-4487-a1a9-4408c37e67ba admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:52.711799 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e00d00-7fb8-4487-a1a9-4408c37e67ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:21:52.714815 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e00d00-7fb8-4487-a1a9-4408c37e67ba admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:21:52.736658 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1309/2607] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:21:52 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:21:52.744748 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f6fd94-fcfe-4fbe-958a-77596a546766 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:21:52.765208 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f6fd94-fcfe-4fbe-958a-77596a546766 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:21:52.774478 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f479b6-5478-42b2-8277-627b3f1f7d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:21:52.775772 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f479b6-5478-42b2-8277-627b3f1f7d4e admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:52.776278 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f479b6-5478-42b2-8277-627b3f1f7d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:52.776774 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f479b6-5478-42b2-8277-627b3f1f7d4e admin admin] PATH_INFO: `/v3/projects/fd29ae3211944daa832e162f69310ff3/users/fd98406ae6b142299e8e25166610fa03/roles/759f389b221049c4b01faf3b984cd8d7` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:52.804942 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f479b6-5478-42b2-8277-627b3f1f7d4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd29ae3211944daa832e162f69310ff3, user_id=fd98406ae6b142299e8e25166610fa03, role_id=759f389b221049c4b01faf3b984cd8d7)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:21:52.809095 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f479b6-5478-42b2-8277-627b3f1f7d4e admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:21:52.838614 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1299/2608] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 14:21:52 2022] PUT /identity/v3/projects/fd29ae3211944daa832e162f69310ff3/users/fd98406ae6b142299e8e25166610fa03/roles/759f389b221049c4b01faf3b984cd8d7 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:21:52.876925 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8697bfc3-7489-4669-8a7d-605f445e5ebb None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:52.877401 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8697bfc3-7489-4669-8a7d-605f445e5ebb None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:52.877921 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8697bfc3-7489-4669-8a7d-605f445e5ebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:52.928825 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-8697bfc3-7489-4669-8a7d-605f445e5ebb None None] MFA Rules not processed for user `fd98406ae6b142299e8e25166610fa03`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:21:52.931412 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-8697bfc3-7489-4669-8a7d-605f445e5ebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:21:53.063964 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1310/2609] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:21:52 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:21:53.083478 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f479b6-5478-42b2-8277-627b3f1f7d4e admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:21:53.085257 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-78f479b6-5478-42b2-8277-627b3f1f7d4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:21:53.203768 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f479b6-5478-42b2-8277-627b3f1f7d4e admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:21:53.209746 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8466d6-1e79-4920-bca0-6903f265c053 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:21:53.210703 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8466d6-1e79-4920-bca0-6903f265c053 service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:53.211162 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8466d6-1e79-4920-bca0-6903f265c053 service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:53.211609 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8466d6-1e79-4920-bca0-6903f265c053 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:53.225755 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8466d6-1e79-4920-bca0-6903f265c053 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:21:53.227206 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8466d6-1e79-4920-bca0-6903f265c053 service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:21:53.237739 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1300/2610] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:21:53 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:21:54.835793 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d1e8b9-cba3-4e83-beea-d9ae584b7f0c None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:54.836616 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d1e8b9-cba3-4e83-beea-d9ae584b7f0c None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:54.836616 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d1e8b9-cba3-4e83-beea-d9ae584b7f0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:54.857230 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-d9d1e8b9-cba3-4e83-beea-d9ae584b7f0c None None] MFA Rules not processed for user `fd98406ae6b142299e8e25166610fa03`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:21:54.859275 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-d9d1e8b9-cba3-4e83-beea-d9ae584b7f0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:21:54.866785 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1311/2611] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:21:54 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:21:55.698962 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8466d6-1e79-4920-bca0-6903f265c053 service cinder] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:21:55.705120 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-dd8466d6-1e79-4920-bca0-6903f265c053 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:21:55.784544 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:21:55.814864 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8466d6-1e79-4920-bca0-6903f265c053 service cinder] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:21:55.821431 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efca1738-8049-4d26-84b8-dc3c899592ae service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '835cbc13756b41b598142df3e35f5492', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:21:55.822127 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efca1738-8049-4d26-84b8-dc3c899592ae service placement] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:21:55.822413 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efca1738-8049-4d26-84b8-dc3c899592ae service placement] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:21:55.822691 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efca1738-8049-4d26-84b8-dc3c899592ae service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:21:55.825769 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-efca1738-8049-4d26-84b8-dc3c899592ae service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:21:55.843622 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efca1738-8049-4d26-84b8-dc3c899592ae service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:21:55.846627 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efca1738-8049-4d26-84b8-dc3c899592ae service placement] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:21:55.864042 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1301/2612] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:21:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:22:04.851022 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=59,keystone:INSERT=3 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:22:05.359993 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d1e8b9-cba3-4e83-beea-d9ae584b7f0c None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:22:05.362246 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-d9d1e8b9-cba3-4e83-beea-d9ae584b7f0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:22:05.478431 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d1e8b9-cba3-4e83-beea-d9ae584b7f0c None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:22:05.487054 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edeffdb-d1a4-4cfd-9697-eb30c3b11dec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c69161cd520b4b29bf50f3bf2f1c033b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:22:05.487685 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edeffdb-d1a4-4cfd-9697-eb30c3b11dec service nova] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:22:05.487967 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edeffdb-d1a4-4cfd-9697-eb30c3b11dec service nova] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:22:05.488240 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edeffdb-d1a4-4cfd-9697-eb30c3b11dec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:22:05.490154 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-5edeffdb-d1a4-4cfd-9697-eb30c3b11dec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:22:05.503037 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edeffdb-d1a4-4cfd-9697-eb30c3b11dec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:22:05.504631 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edeffdb-d1a4-4cfd-9697-eb30c3b11dec service nova] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:22:05.516752 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1312/2613] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:22:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:22:05.858946 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=11 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:22:06.395626 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efca1738-8049-4d26-84b8-dc3c899592ae service placement] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:22:06.397578 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-efca1738-8049-4d26-84b8-dc3c899592ae service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:22:06.483759 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efca1738-8049-4d26-84b8-dc3c899592ae service placement] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:22:06.490498 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844bde34-10ed-499a-88c2-1ff24054daaa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:22:06.491644 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844bde34-10ed-499a-88c2-1ff24054daaa service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:22:06.492142 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844bde34-10ed-499a-88c2-1ff24054daaa service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:22:06.492602 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844bde34-10ed-499a-88c2-1ff24054daaa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:22:06.495432 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-844bde34-10ed-499a-88c2-1ff24054daaa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:22:06.597426 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844bde34-10ed-499a-88c2-1ff24054daaa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:22:06.599295 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844bde34-10ed-499a-88c2-1ff24054daaa service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:22:06.612704 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1302/2614] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:22:06 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 14:22:12.350482 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edeffdb-d1a4-4cfd-9697-eb30c3b11dec service nova] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:22:12.371293 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edeffdb-d1a4-4cfd-9697-eb30c3b11dec service nova] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:22:12.380676 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db60d3e-bbd4-47f2-85b0-f6a90c751abf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c69161cd520b4b29bf50f3bf2f1c033b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:22:12.381598 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db60d3e-bbd4-47f2-85b0-f6a90c751abf service nova] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:22:12.382103 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db60d3e-bbd4-47f2-85b0-f6a90c751abf service nova] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:22:12.382557 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db60d3e-bbd4-47f2-85b0-f6a90c751abf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:22:12.385613 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-8db60d3e-bbd4-47f2-85b0-f6a90c751abf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:22:12.400289 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db60d3e-bbd4-47f2-85b0-f6a90c751abf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:22:12.402976 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db60d3e-bbd4-47f2-85b0-f6a90c751abf service nova] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:22:12.417278 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1313/2615] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:22:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:22:13.274294 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c921c05c-1979-43e9-8f77-fb0acf192454 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:22:13.274829 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c921c05c-1979-43e9-8f77-fb0acf192454 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:22:13.274829 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c921c05c-1979-43e9-8f77-fb0acf192454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:22:13.297073 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-c921c05c-1979-43e9-8f77-fb0acf192454 None None] MFA Rules not processed for user `b64bd7b183d24d8c9037b261c7884810`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:22:13.299208 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-c921c05c-1979-43e9-8f77-fb0acf192454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:22:13.374649 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1303/2616] 10.222.0.58 () {58 vars in 1009 bytes} [Tue Jul 26 14:22:13 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:22:13.388231 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db60d3e-bbd4-47f2-85b0-f6a90c751abf service nova] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:22:13.389846 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-8db60d3e-bbd4-47f2-85b0-f6a90c751abf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:22:13.481744 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db60d3e-bbd4-47f2-85b0-f6a90c751abf service nova] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:22:13.487467 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafd9a38-086f-4f75-b4cf-bfbfeb83ce2d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab2819bac844163ae4305e4f2186605', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:22:13.488241 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafd9a38-086f-4f75-b4cf-bfbfeb83ce2d service swift] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:22:13.488402 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafd9a38-086f-4f75-b4cf-bfbfeb83ce2d service swift] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:22:13.488620 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafd9a38-086f-4f75-b4cf-bfbfeb83ce2d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:22:13.502343 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafd9a38-086f-4f75-b4cf-bfbfeb83ce2d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:22:13.505567 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafd9a38-086f-4f75-b4cf-bfbfeb83ce2d service swift] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:22:13.520173 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1314/2617] 10.222.0.58 () {60 vars in 1384 bytes} [Tue Jul 26 14:22:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 14:22:14.185849 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776272b7-62cd-49b9-b4df-1b5ec9c9a49d None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:22:14.186290 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776272b7-62cd-49b9-b4df-1b5ec9c9a49d None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:22:14.186633 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776272b7-62cd-49b9-b4df-1b5ec9c9a49d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:22:14.207344 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-776272b7-62cd-49b9-b4df-1b5ec9c9a49d None None] MFA Rules not processed for user `b64bd7b183d24d8c9037b261c7884810`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:22:14.208562 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-776272b7-62cd-49b9-b4df-1b5ec9c9a49d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:22:14.216518 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1304/2618] 10.222.0.58 () {58 vars in 967 bytes} [Tue Jul 26 14:22:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:22:14.230134 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafd9a38-086f-4f75-b4cf-bfbfeb83ce2d service swift] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:22:14.243319 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafd9a38-086f-4f75-b4cf-bfbfeb83ce2d service swift] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:22:14.251823 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932e5b72-ff7c-428f-9286-6cf6e25f0d33 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab2819bac844163ae4305e4f2186605', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:22:14.252878 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932e5b72-ff7c-428f-9286-6cf6e25f0d33 service swift] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:22:14.253393 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932e5b72-ff7c-428f-9286-6cf6e25f0d33 service swift] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:22:14.253932 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932e5b72-ff7c-428f-9286-6cf6e25f0d33 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:22:14.264270 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932e5b72-ff7c-428f-9286-6cf6e25f0d33 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:22:14.265925 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932e5b72-ff7c-428f-9286-6cf6e25f0d33 service swift] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:22:14.277925 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1315/2619] 10.222.0.58 () {60 vars in 1384 bytes} [Tue Jul 26 14:22:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 14:22:14.563297 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b821520f-e0d7-47e3-a88f-45c46af67f0b None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:22:14.563887 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b821520f-e0d7-47e3-a88f-45c46af67f0b None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:22:14.564087 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b821520f-e0d7-47e3-a88f-45c46af67f0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:22:14.594773 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-b821520f-e0d7-47e3-a88f-45c46af67f0b None None] MFA Rules not processed for user `b64bd7b183d24d8c9037b261c7884810`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:22:14.596803 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-b821520f-e0d7-47e3-a88f-45c46af67f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:22:14.604903 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1305/2620] 10.222.0.58 () {58 vars in 1009 bytes} [Tue Jul 26 14:22:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:22:14.618176 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932e5b72-ff7c-428f-9286-6cf6e25f0d33 service swift] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:22:14.632319 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932e5b72-ff7c-428f-9286-6cf6e25f0d33 service swift] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:22:14.642287 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d716b7f6-8ceb-41d3-843d-beb5d9df0b79 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab2819bac844163ae4305e4f2186605', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:22:14.643318 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d716b7f6-8ceb-41d3-843d-beb5d9df0b79 service swift] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:22:14.643808 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d716b7f6-8ceb-41d3-843d-beb5d9df0b79 service swift] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:22:14.644244 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d716b7f6-8ceb-41d3-843d-beb5d9df0b79 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:22:14.659865 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d716b7f6-8ceb-41d3-843d-beb5d9df0b79 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:22:14.662719 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d716b7f6-8ceb-41d3-843d-beb5d9df0b79 service swift] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:22:14.678429 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1316/2621] 10.222.0.58 () {60 vars in 1384 bytes} [Tue Jul 26 14:22:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 14:22:15.161125 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676fe8d6-da6e-47ff-aa86-4e9273d86bcc None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:22:15.161712 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676fe8d6-da6e-47ff-aa86-4e9273d86bcc None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:22:15.162000 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676fe8d6-da6e-47ff-aa86-4e9273d86bcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:22:15.188897 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-676fe8d6-da6e-47ff-aa86-4e9273d86bcc None None] MFA Rules not processed for user `b64bd7b183d24d8c9037b261c7884810`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:22:15.191026 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-676fe8d6-da6e-47ff-aa86-4e9273d86bcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:22:15.201696 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1306/2622] 10.222.0.58 () {58 vars in 967 bytes} [Tue Jul 26 14:22:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:22:15.215810 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d716b7f6-8ceb-41d3-843d-beb5d9df0b79 service swift] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:22:15.229334 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d716b7f6-8ceb-41d3-843d-beb5d9df0b79 service swift] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:22:15.236310 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e6bb9c-1fc5-403c-9c58-7c4c2fcbeab2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab2819bac844163ae4305e4f2186605', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:22:15.237066 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e6bb9c-1fc5-403c-9c58-7c4c2fcbeab2 service swift] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:22:15.237384 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e6bb9c-1fc5-403c-9c58-7c4c2fcbeab2 service swift] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:22:15.237825 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e6bb9c-1fc5-403c-9c58-7c4c2fcbeab2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:22:15.253211 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e6bb9c-1fc5-403c-9c58-7c4c2fcbeab2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:22:15.256325 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e6bb9c-1fc5-403c-9c58-7c4c2fcbeab2 service swift] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:22:15.272710 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1317/2623] 10.222.0.58 () {60 vars in 1384 bytes} [Tue Jul 26 14:22:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 26 14:22:17.756275 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676fe8d6-da6e-47ff-aa86-4e9273d86bcc None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:22:17.811692 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:22:17.840814 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676fe8d6-da6e-47ff-aa86-4e9273d86bcc None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:22:17.845895 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145afe36-18b1-49ae-a649-3f28dadca251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:22:17.846576 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145afe36-18b1-49ae-a649-3f28dadca251 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:22:17.846854 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145afe36-18b1-49ae-a649-3f28dadca251 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:22:17.847133 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145afe36-18b1-49ae-a649-3f28dadca251 admin admin] PATH_INFO: `/v3/users/fd98406ae6b142299e8e25166610fa03` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:22:17.848355 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145afe36-18b1-49ae-a649-3f28dadca251 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd98406ae6b142299e8e25166610fa03)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:22:17.850342 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145afe36-18b1-49ae-a649-3f28dadca251 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:22:17.937484 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-145afe36-18b1-49ae-a649-3f28dadca251 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd98406ae6b142299e8e25166610fa03'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:22:17.938785 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-145afe36-18b1-49ae-a649-3f28dadca251 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd98406ae6b142299e8e25166610fa03'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:22:17.951296 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-145afe36-18b1-49ae-a649-3f28dadca251 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd98406ae6b142299e8e25166610fa03'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:22:17.952245 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-145afe36-18b1-49ae-a649-3f28dadca251 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd98406ae6b142299e8e25166610fa03'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:22:17.971250 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-145afe36-18b1-49ae-a649-3f28dadca251 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd98406ae6b142299e8e25166610fa03'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:22:17.990848 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1307/2624] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 14:22:17 2022] DELETE /identity/v3/users/fd98406ae6b142299e8e25166610fa03 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:22:17.996809 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eea892d-4738-4630-9b4f-4f916da23ce7 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:22:17.997393 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eea892d-4738-4630-9b4f-4f916da23ce7 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:22:17.997563 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eea892d-4738-4630-9b4f-4f916da23ce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:22:18.021139 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-2eea892d-4738-4630-9b4f-4f916da23ce7 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:22:18.022386 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-2eea892d-4738-4630-9b4f-4f916da23ce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:22:18.049137 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:22:18.125121 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1318/2625] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:22:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:22:18.138024 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145afe36-18b1-49ae-a649-3f28dadca251 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:22:18.140550 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-145afe36-18b1-49ae-a649-3f28dadca251 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:22:18.215170 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145afe36-18b1-49ae-a649-3f28dadca251 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:22:18.218889 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358cfb7d-f282-4f5f-9642-173afcab250b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:22:18.219525 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358cfb7d-f282-4f5f-9642-173afcab250b service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:22:18.219814 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358cfb7d-f282-4f5f-9642-173afcab250b service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:22:18.220090 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358cfb7d-f282-4f5f-9642-173afcab250b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:22:18.230464 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358cfb7d-f282-4f5f-9642-173afcab250b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:22:18.231900 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358cfb7d-f282-4f5f-9642-173afcab250b service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:22:18.247131 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1308/2626] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:22:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:22:18.486443 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eea892d-4738-4630-9b4f-4f916da23ce7 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:22:18.488814 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-2eea892d-4738-4630-9b4f-4f916da23ce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:22:18.506166 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eea892d-4738-4630-9b4f-4f916da23ce7 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:22:18.513241 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5de4117-1939-4d48-adcd-82d12dd26038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:22:18.514810 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5de4117-1939-4d48-adcd-82d12dd26038 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:22:18.515161 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5de4117-1939-4d48-adcd-82d12dd26038 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:22:18.515344 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5de4117-1939-4d48-adcd-82d12dd26038 admin admin] PATH_INFO: `/v3/projects/fd29ae3211944daa832e162f69310ff3` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:22:18.516786 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5de4117-1939-4d48-adcd-82d12dd26038 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd29ae3211944daa832e162f69310ff3)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:22:18.519481 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5de4117-1939-4d48-adcd-82d12dd26038 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:22:18.589305 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-c5de4117-1939-4d48-adcd-82d12dd26038 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd29ae3211944daa832e162f69310ff3', 'request_id': 'req-c5de4117-1939-4d48-adcd-82d12dd26038'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:22:18.598256 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-c5de4117-1939-4d48-adcd-82d12dd26038 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd29ae3211944daa832e162f69310ff3', 'request_id': 'req-c5de4117-1939-4d48-adcd-82d12dd26038'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:22:18.605808 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-c5de4117-1939-4d48-adcd-82d12dd26038 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd29ae3211944daa832e162f69310ff3', 'request_id': 'req-c5de4117-1939-4d48-adcd-82d12dd26038'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:22:18.619993 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-c5de4117-1939-4d48-adcd-82d12dd26038 admin admin] The token cache is being invalidate because project fd29ae3211944daa832e162f69310ff3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86933) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:22:18.620329 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-c5de4117-1939-4d48-adcd-82d12dd26038 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:22:18.620953 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-c5de4117-1939-4d48-adcd-82d12dd26038 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:22:18.622570 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1319/2627] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 14:22:18 2022] DELETE /identity/v3/projects/fd29ae3211944daa832e162f69310ff3 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:22:18.628459 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c49950-5c5e-4a79-bcbc-35117608a83d None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:22:18.628842 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c49950-5c5e-4a79-bcbc-35117608a83d None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:22:18.629194 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c49950-5c5e-4a79-bcbc-35117608a83d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:22:18.658015 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-54c49950-5c5e-4a79-bcbc-35117608a83d None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:22:18.659545 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-54c49950-5c5e-4a79-bcbc-35117608a83d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:22:18.726779 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:22:18.742207 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1309/2628] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:22:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:22:18.753224 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a892aff-1658-4f6c-b337-384a2e700b47 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:22:18.753735 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a892aff-1658-4f6c-b337-384a2e700b47 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:22:18.754240 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a892aff-1658-4f6c-b337-384a2e700b47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:22:18.787604 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-0a892aff-1658-4f6c-b337-384a2e700b47 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:22:18.789749 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-0a892aff-1658-4f6c-b337-384a2e700b47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:22:18.803766 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1320/2629] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:22:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:22:18.810490 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c49950-5c5e-4a79-bcbc-35117608a83d None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:22:18.821179 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c49950-5c5e-4a79-bcbc-35117608a83d None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:22:18.826343 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835cec75-de1d-4c8e-be9d-0f6f8b1e26ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:22:18.827011 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835cec75-de1d-4c8e-be9d-0f6f8b1e26ed admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:22:18.827309 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835cec75-de1d-4c8e-be9d-0f6f8b1e26ed admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:22:18.827582 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835cec75-de1d-4c8e-be9d-0f6f8b1e26ed admin admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:22:18.828325 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835cec75-de1d-4c8e-be9d-0f6f8b1e26ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:22:18.828634 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835cec75-de1d-4c8e-be9d-0f6f8b1e26ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:22:18.830293 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835cec75-de1d-4c8e-be9d-0f6f8b1e26ed admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:22:18.843818 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1310/2630] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Jul 26 14:22:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:22:18.850136 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a892aff-1658-4f6c-b337-384a2e700b47 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:22:18.862538 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a892aff-1658-4f6c-b337-384a2e700b47 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:22:18.868089 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991c701f-9147-4f55-a5b6-7fd25d457293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:22:18.868939 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991c701f-9147-4f55-a5b6-7fd25d457293 admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:22:18.870017 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991c701f-9147-4f55-a5b6-7fd25d457293 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:22:18.870499 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991c701f-9147-4f55-a5b6-7fd25d457293 admin admin] PATH_INFO: `/v3/projects` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:22:18.871435 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991c701f-9147-4f55-a5b6-7fd25d457293 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:22:18.874083 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991c701f-9147-4f55-a5b6-7fd25d457293 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:22:18.907962 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1321/2631] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 14:22:18 2022] POST /identity/v3/projects => generated 377 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:22:18.913439 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835cec75-de1d-4c8e-be9d-0f6f8b1e26ed admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:22:18.932173 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835cec75-de1d-4c8e-be9d-0f6f8b1e26ed admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:22:19.177029 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653466de-7f2e-458e-8220-cb6b856b5589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:22:19.177831 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653466de-7f2e-458e-8220-cb6b856b5589 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:22:19.178113 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653466de-7f2e-458e-8220-cb6b856b5589 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:22:19.178387 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653466de-7f2e-458e-8220-cb6b856b5589 admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:22:19.179100 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653466de-7f2e-458e-8220-cb6b856b5589 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:22:19.181040 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653466de-7f2e-458e-8220-cb6b856b5589 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:22:19.206188 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1311/2632] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 14:22:18 2022] POST /identity/v3/users => generated 289 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:22:19.211848 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991c701f-9147-4f55-a5b6-7fd25d457293 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:22:19.222630 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991c701f-9147-4f55-a5b6-7fd25d457293 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:22:19.227245 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236f8296-eb6e-4d30-acac-ecaed492b1d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:22:19.227877 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236f8296-eb6e-4d30-acac-ecaed492b1d0 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:22:19.228161 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236f8296-eb6e-4d30-acac-ecaed492b1d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:22:19.228436 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236f8296-eb6e-4d30-acac-ecaed492b1d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:22:19.229193 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236f8296-eb6e-4d30-acac-ecaed492b1d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:22:19.230845 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236f8296-eb6e-4d30-acac-ecaed492b1d0 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:22:19.241465 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1322/2633] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:22:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:22:19.246412 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653466de-7f2e-458e-8220-cb6b856b5589 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:22:19.257662 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653466de-7f2e-458e-8220-cb6b856b5589 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:22:19.263390 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adafb914-97ac-4857-81c1-10aa79d6bf93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:22:19.264017 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adafb914-97ac-4857-81c1-10aa79d6bf93 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:22:19.264293 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adafb914-97ac-4857-81c1-10aa79d6bf93 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:22:19.264568 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adafb914-97ac-4857-81c1-10aa79d6bf93 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:22:19.265264 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adafb914-97ac-4857-81c1-10aa79d6bf93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:22:19.266887 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adafb914-97ac-4857-81c1-10aa79d6bf93 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:22:19.280709 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1312/2634] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:22:19 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 14:22:19.285352 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236f8296-eb6e-4d30-acac-ecaed492b1d0 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:22:19.297262 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236f8296-eb6e-4d30-acac-ecaed492b1d0 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:22:19.303055 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ab1908-97ef-49aa-9403-de324df05c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:22:19.303996 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ab1908-97ef-49aa-9403-de324df05c0f admin admin] REQUEST_METHOD: `PUT` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:22:19.304378 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ab1908-97ef-49aa-9403-de324df05c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:22:19.304831 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ab1908-97ef-49aa-9403-de324df05c0f admin admin] PATH_INFO: `/v3/projects/35c9722406c24d7982e271119af624d7/users/7745969be55b487dbb15b216a3016731/roles/759f389b221049c4b01faf3b984cd8d7` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:22:19.326830 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ab1908-97ef-49aa-9403-de324df05c0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=35c9722406c24d7982e271119af624d7, user_id=7745969be55b487dbb15b216a3016731, role_id=759f389b221049c4b01faf3b984cd8d7)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:22:19.332141 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ab1908-97ef-49aa-9403-de324df05c0f admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:22:19.355433 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1323/2635] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 14:22:19 2022] PUT /identity/v3/projects/35c9722406c24d7982e271119af624d7/users/7745969be55b487dbb15b216a3016731/roles/759f389b221049c4b01faf3b984cd8d7 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:22:19.361118 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142021d9-34b0-422c-95e9-5e8dc891077e None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:22:19.361454 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142021d9-34b0-422c-95e9-5e8dc891077e None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:22:19.361838 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142021d9-34b0-422c-95e9-5e8dc891077e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:22:19.382496 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-142021d9-34b0-422c-95e9-5e8dc891077e None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:22:19.383629 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-142021d9-34b0-422c-95e9-5e8dc891077e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:22:19.455772 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1313/2636] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:22:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:22:19.466813 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ab1908-97ef-49aa-9403-de324df05c0f admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:22:19.468682 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-23ab1908-97ef-49aa-9403-de324df05c0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:22:19.518098 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:22:19.539461 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ab1908-97ef-49aa-9403-de324df05c0f admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:22:19.544583 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b36bd84-7c82-40ee-90a2-e913b8c3baf3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:22:19.545351 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b36bd84-7c82-40ee-90a2-e913b8c3baf3 service neutron] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:22:19.545717 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b36bd84-7c82-40ee-90a2-e913b8c3baf3 service neutron] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:22:19.546058 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b36bd84-7c82-40ee-90a2-e913b8c3baf3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:22:19.562704 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b36bd84-7c82-40ee-90a2-e913b8c3baf3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:22:19.564159 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b36bd84-7c82-40ee-90a2-e913b8c3baf3 service neutron] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:22:19.575880 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1324/2637] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:22:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:22:24.807756 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34235237-0b82-4fce-9655-b1148f6b1c2d None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:22:24.808305 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34235237-0b82-4fce-9655-b1148f6b1c2d None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:22:24.808421 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34235237-0b82-4fce-9655-b1148f6b1c2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:22:24.850745 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-34235237-0b82-4fce-9655-b1148f6b1c2d None None] MFA Rules not processed for user `7745969be55b487dbb15b216a3016731`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:22:24.852917 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-34235237-0b82-4fce-9655-b1148f6b1c2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:22:24.923296 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:22:24.941727 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1314/2638] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:22:24 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:22:24.943860 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b36bd84-7c82-40ee-90a2-e913b8c3baf3 service neutron] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:22:24.963366 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b36bd84-7c82-40ee-90a2-e913b8c3baf3 service neutron] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:22:24.970103 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedbce4e-43ba-4402-8828-264f6d382121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:22:24.971142 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedbce4e-43ba-4402-8828-264f6d382121 admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:22:24.971614 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedbce4e-43ba-4402-8828-264f6d382121 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:22:24.972107 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedbce4e-43ba-4402-8828-264f6d382121 admin admin] PATH_INFO: `/v3/projects` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:22:24.973015 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedbce4e-43ba-4402-8828-264f6d382121 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:22:24.976522 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedbce4e-43ba-4402-8828-264f6d382121 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:22:25.025152 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1325/2639] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 14:22:24 2022] POST /identity/v3/projects => generated 377 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 14:22:25.031798 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34235237-0b82-4fce-9655-b1148f6b1c2d None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:22:25.050270 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34235237-0b82-4fce-9655-b1148f6b1c2d None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:22:25.148769 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27011471-9b21-47d6-99e5-2edba48764b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:22:25.150028 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27011471-9b21-47d6-99e5-2edba48764b1 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:22:25.151132 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27011471-9b21-47d6-99e5-2edba48764b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:22:25.151716 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27011471-9b21-47d6-99e5-2edba48764b1 admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:22:25.152985 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27011471-9b21-47d6-99e5-2edba48764b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:22:25.157142 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27011471-9b21-47d6-99e5-2edba48764b1 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:22:25.191587 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1315/2640] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 14:22:25 2022] POST /identity/v3/users => generated 295 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:22:25.197539 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedbce4e-43ba-4402-8828-264f6d382121 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:22:25.217162 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedbce4e-43ba-4402-8828-264f6d382121 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:22:25.224725 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1d18cc-d5f3-4bbb-8ffc-dc727fb2ec5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:22:25.225810 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1d18cc-d5f3-4bbb-8ffc-dc727fb2ec5c admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:22:25.226277 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1d18cc-d5f3-4bbb-8ffc-dc727fb2ec5c admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:22:25.226694 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1d18cc-d5f3-4bbb-8ffc-dc727fb2ec5c admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:22:25.227864 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1d18cc-d5f3-4bbb-8ffc-dc727fb2ec5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:22:25.231155 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1d18cc-d5f3-4bbb-8ffc-dc727fb2ec5c admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:22:25.250070 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1326/2641] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:22:25 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:22:25.254812 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27011471-9b21-47d6-99e5-2edba48764b1 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:22:25.269688 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27011471-9b21-47d6-99e5-2edba48764b1 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:22:25.277993 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dfdc9b-f2ad-4ec2-bcd0-39162f5ff06f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:22:25.279173 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dfdc9b-f2ad-4ec2-bcd0-39162f5ff06f admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:22:25.279695 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dfdc9b-f2ad-4ec2-bcd0-39162f5ff06f admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:22:25.280158 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dfdc9b-f2ad-4ec2-bcd0-39162f5ff06f admin admin] PATH_INFO: `/v3/projects/c87c169bd8dd4c4797c07187e0fecdc4/users/9fd120bb45394b8984de9455dc7d3841/roles/6e0201a9a6484f7d8b112b1eafff70bd` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:22:25.327856 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dfdc9b-f2ad-4ec2-bcd0-39162f5ff06f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c87c169bd8dd4c4797c07187e0fecdc4, user_id=9fd120bb45394b8984de9455dc7d3841, role_id=6e0201a9a6484f7d8b112b1eafff70bd)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:22:25.331183 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dfdc9b-f2ad-4ec2-bcd0-39162f5ff06f admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:22:25.352247 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1316/2642] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 14:22:25 2022] PUT /identity/v3/projects/c87c169bd8dd4c4797c07187e0fecdc4/users/9fd120bb45394b8984de9455dc7d3841/roles/6e0201a9a6484f7d8b112b1eafff70bd => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:22:25.357451 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1d18cc-d5f3-4bbb-8ffc-dc727fb2ec5c admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:22:25.375202 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1d18cc-d5f3-4bbb-8ffc-dc727fb2ec5c admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:22:25.483589 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1609196b-9eb2-40bd-8734-124f2c0b8957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:22:25.484598 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1609196b-9eb2-40bd-8734-124f2c0b8957 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:22:25.485065 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1609196b-9eb2-40bd-8734-124f2c0b8957 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:22:25.485583 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1609196b-9eb2-40bd-8734-124f2c0b8957 admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:22:25.486755 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1609196b-9eb2-40bd-8734-124f2c0b8957 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:22:25.489424 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1609196b-9eb2-40bd-8734-124f2c0b8957 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:22:25.503868 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1327/2643] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:22:25 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:22:25.510622 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dfdc9b-f2ad-4ec2-bcd0-39162f5ff06f admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:22:25.527832 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dfdc9b-f2ad-4ec2-bcd0-39162f5ff06f admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:22:25.535931 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7476282-fdea-4e40-b58b-f8112c26bc84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:22:25.537102 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7476282-fdea-4e40-b58b-f8112c26bc84 admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:22:25.537637 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7476282-fdea-4e40-b58b-f8112c26bc84 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:22:25.538128 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7476282-fdea-4e40-b58b-f8112c26bc84 admin admin] PATH_INFO: `/v3/projects/c87c169bd8dd4c4797c07187e0fecdc4/users/9fd120bb45394b8984de9455dc7d3841/roles/6e0201a9a6484f7d8b112b1eafff70bd` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:22:25.542420 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7476282-fdea-4e40-b58b-f8112c26bc84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c87c169bd8dd4c4797c07187e0fecdc4, user_id=9fd120bb45394b8984de9455dc7d3841, role_id=6e0201a9a6484f7d8b112b1eafff70bd)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:22:25.547568 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7476282-fdea-4e40-b58b-f8112c26bc84 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:22:25.566909 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1317/2644] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 14:22:25 2022] PUT /identity/v3/projects/c87c169bd8dd4c4797c07187e0fecdc4/users/9fd120bb45394b8984de9455dc7d3841/roles/6e0201a9a6484f7d8b112b1eafff70bd => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:22:32.446657 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6346ce-b369-4070-816d-f6a90accb467 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:22:32.448185 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6346ce-b369-4070-816d-f6a90accb467 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:22:32.448639 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6346ce-b369-4070-816d-f6a90accb467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:22:32.481399 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-6b6346ce-b369-4070-816d-f6a90accb467 None None] MFA Rules not processed for user `9fd120bb45394b8984de9455dc7d3841`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:22:32.482696 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-6b6346ce-b369-4070-816d-f6a90accb467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:22:32.602196 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1328/2645] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:22:32 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:22:32.637500 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7476282-fdea-4e40-b58b-f8112c26bc84 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:22:32.644309 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-c7476282-fdea-4e40-b58b-f8112c26bc84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:22:32.795512 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7476282-fdea-4e40-b58b-f8112c26bc84 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:22:32.805603 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa8731e-e59f-447a-a552-5fd7e79c20d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c69161cd520b4b29bf50f3bf2f1c033b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:22:32.806758 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa8731e-e59f-447a-a552-5fd7e79c20d0 service nova] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:22:32.807635 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa8731e-e59f-447a-a552-5fd7e79c20d0 service nova] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:22:32.808081 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa8731e-e59f-447a-a552-5fd7e79c20d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:22:32.873074 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:22:32.896309 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa8731e-e59f-447a-a552-5fd7e79c20d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:22:32.898248 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa8731e-e59f-447a-a552-5fd7e79c20d0 service nova] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:22:32.909991 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1318/2646] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:22:32 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:22:42.596479 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=84,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:22:42.904924 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=8 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:22:54.779155 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f093d05-fe08-4b21-9014-2a120551d9e3 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:22:54.779878 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f093d05-fe08-4b21-9014-2a120551d9e3 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:22:54.779878 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f093d05-fe08-4b21-9014-2a120551d9e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:22:54.799804 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-5f093d05-fe08-4b21-9014-2a120551d9e3 None None] MFA Rules not processed for user `7745969be55b487dbb15b216a3016731`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:22:54.801092 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-5f093d05-fe08-4b21-9014-2a120551d9e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:22:54.811730 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1329/2647] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:22:54 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:22:55.244680 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa8731e-e59f-447a-a552-5fd7e79c20d0 service nova] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:22:55.246600 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-5fa8731e-e59f-447a-a552-5fd7e79c20d0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:22:55.335544 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa8731e-e59f-447a-a552-5fd7e79c20d0 service nova] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:22:55.340031 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c40a0d1-c82c-4be2-88bc-8a6e2d6578b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:22:55.340774 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c40a0d1-c82c-4be2-88bc-8a6e2d6578b4 service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:22:55.341074 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c40a0d1-c82c-4be2-88bc-8a6e2d6578b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:22:55.341353 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c40a0d1-c82c-4be2-88bc-8a6e2d6578b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:22:55.343711 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-4c40a0d1-c82c-4be2-88bc-8a6e2d6578b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:22:55.358489 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c40a0d1-c82c-4be2-88bc-8a6e2d6578b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:22:55.359955 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c40a0d1-c82c-4be2-88bc-8a6e2d6578b4 service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:22:55.370656 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1319/2648] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:22:55 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:23:04.794004 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=3 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:23:05.365331 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=31 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:23:12.620184 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f093d05-fe08-4b21-9014-2a120551d9e3 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:23:12.631348 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f093d05-fe08-4b21-9014-2a120551d9e3 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:23:12.635523 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dd51e7-c020-4583-8341-d222379575b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:23:12.636417 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dd51e7-c020-4583-8341-d222379575b6 service neutron] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:23:12.636701 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dd51e7-c020-4583-8341-d222379575b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:23:12.636972 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dd51e7-c020-4583-8341-d222379575b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:23:12.647036 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dd51e7-c020-4583-8341-d222379575b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:23:12.648502 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dd51e7-c020-4583-8341-d222379575b6 service neutron] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:23:12.660952 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1330/2649] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:23:12 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:23:22.654201 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=6 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:23:43.974459 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c40a0d1-c82c-4be2-88bc-8a6e2d6578b4 service neutron] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:23:43.976013 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-4c40a0d1-c82c-4be2-88bc-8a6e2d6578b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:23:44.049554 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c40a0d1-c82c-4be2-88bc-8a6e2d6578b4 service neutron] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:23:44.054313 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b9dda1-d78c-43a3-943f-86a573ffafe6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:23:44.055067 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b9dda1-d78c-43a3-943f-86a573ffafe6 service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:23:44.055396 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b9dda1-d78c-43a3-943f-86a573ffafe6 service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:23:44.055713 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b9dda1-d78c-43a3-943f-86a573ffafe6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:23:44.057745 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-a7b9dda1-d78c-43a3-943f-86a573ffafe6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:23:44.133321 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b9dda1-d78c-43a3-943f-86a573ffafe6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:23:44.135009 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b9dda1-d78c-43a3-943f-86a573ffafe6 service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:23:44.145532 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1320/2650] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:23:43 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:23:54.141435 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=53 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:24:00.871314 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909d78f4-fce2-427d-93dc-2273b5d406c5 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:24:00.871838 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909d78f4-fce2-427d-93dc-2273b5d406c5 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:24:00.871838 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909d78f4-fce2-427d-93dc-2273b5d406c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:24:00.890996 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-909d78f4-fce2-427d-93dc-2273b5d406c5 None None] MFA Rules not processed for user `b64bd7b183d24d8c9037b261c7884810`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:24:00.892186 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-909d78f4-fce2-427d-93dc-2273b5d406c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:24:00.946493 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1331/2651] 10.222.0.58 () {58 vars in 967 bytes} [Tue Jul 26 14:24:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:24:00.991796 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b9dda1-d78c-43a3-943f-86a573ffafe6 service cinder] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:24:00.993188 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-a7b9dda1-d78c-43a3-943f-86a573ffafe6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:24:01.066676 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b9dda1-d78c-43a3-943f-86a573ffafe6 service cinder] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:24:01.071611 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c04980-0183-4e77-8cdc-9041923eb135 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab2819bac844163ae4305e4f2186605', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:24:01.072232 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c04980-0183-4e77-8cdc-9041923eb135 service swift] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:24:01.072511 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c04980-0183-4e77-8cdc-9041923eb135 service swift] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:24:01.072786 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c04980-0183-4e77-8cdc-9041923eb135 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:24:01.080750 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c04980-0183-4e77-8cdc-9041923eb135 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:24:01.082338 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c04980-0183-4e77-8cdc-9041923eb135 service swift] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:24:01.090877 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1321/2652] 10.222.0.58 () {60 vars in 1384 bytes} [Tue Jul 26 14:24:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 14:24:08.011541 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909d78f4-fce2-427d-93dc-2273b5d406c5 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:24:08.013039 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-909d78f4-fce2-427d-93dc-2273b5d406c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:24:08.099957 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909d78f4-fce2-427d-93dc-2273b5d406c5 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:24:08.104531 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0297ac5b-2480-41da-ab70-392bfd3a9586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:24:08.105726 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0297ac5b-2480-41da-ab70-392bfd3a9586 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:24:08.106004 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0297ac5b-2480-41da-ab70-392bfd3a9586 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:24:08.109783 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0297ac5b-2480-41da-ab70-392bfd3a9586 admin admin] PATH_INFO: `/v3/users/02fbfb4bcab74a3c828335ff923616a2` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:24:08.109783 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0297ac5b-2480-41da-ab70-392bfd3a9586 admin admin] RBAC: Authorizing `identity:delete_user(user_id=02fbfb4bcab74a3c828335ff923616a2)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:24:08.109783 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0297ac5b-2480-41da-ab70-392bfd3a9586 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:24:08.157607 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-0297ac5b-2480-41da-ab70-392bfd3a9586 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02fbfb4bcab74a3c828335ff923616a2'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:24:08.166050 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-0297ac5b-2480-41da-ab70-392bfd3a9586 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02fbfb4bcab74a3c828335ff923616a2'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:24:08.173536 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-0297ac5b-2480-41da-ab70-392bfd3a9586 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02fbfb4bcab74a3c828335ff923616a2'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:24:08.174156 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-0297ac5b-2480-41da-ab70-392bfd3a9586 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02fbfb4bcab74a3c828335ff923616a2'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:24:08.174766 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-0297ac5b-2480-41da-ab70-392bfd3a9586 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02fbfb4bcab74a3c828335ff923616a2'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:24:08.205874 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1332/2653] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 14:24:08 2022] DELETE /identity/v3/users/02fbfb4bcab74a3c828335ff923616a2 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:24:08.391457 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c04980-0183-4e77-8cdc-9041923eb135 service swift] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:24:08.403858 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c04980-0183-4e77-8cdc-9041923eb135 service swift] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:24:08.489133 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6303437a-9d2a-4ec0-b796-784b4e9f2c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:24:08.489924 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6303437a-9d2a-4ec0-b796-784b4e9f2c33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:24:08.490271 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6303437a-9d2a-4ec0-b796-784b4e9f2c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:24:08.490731 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6303437a-9d2a-4ec0-b796-784b4e9f2c33 admin admin] PATH_INFO: `/v3/projects/1599897c15514b80bdc9374dd5aa9f64` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:24:08.492906 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6303437a-9d2a-4ec0-b796-784b4e9f2c33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1599897c15514b80bdc9374dd5aa9f64)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:24:08.494896 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6303437a-9d2a-4ec0-b796-784b4e9f2c33 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:24:08.549064 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-6303437a-9d2a-4ec0-b796-784b4e9f2c33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1599897c15514b80bdc9374dd5aa9f64', 'request_id': 'req-6303437a-9d2a-4ec0-b796-784b4e9f2c33'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:24:08.558493 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-6303437a-9d2a-4ec0-b796-784b4e9f2c33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1599897c15514b80bdc9374dd5aa9f64', 'request_id': 'req-6303437a-9d2a-4ec0-b796-784b4e9f2c33'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:24:08.564727 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-6303437a-9d2a-4ec0-b796-784b4e9f2c33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1599897c15514b80bdc9374dd5aa9f64', 'request_id': 'req-6303437a-9d2a-4ec0-b796-784b4e9f2c33'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:24:08.578452 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-6303437a-9d2a-4ec0-b796-784b4e9f2c33 admin admin] The token cache is being invalidate because project 1599897c15514b80bdc9374dd5aa9f64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:24:08.578745 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-6303437a-9d2a-4ec0-b796-784b4e9f2c33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:24:08.579395 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-6303437a-9d2a-4ec0-b796-784b4e9f2c33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:24:08.580912 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1322/2654] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 14:24:08 2022] DELETE /identity/v3/projects/1599897c15514b80bdc9374dd5aa9f64 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:24:08.584983 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0297ac5b-2480-41da-ab70-392bfd3a9586 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:24:08.586527 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-0297ac5b-2480-41da-ab70-392bfd3a9586 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:24:08.600553 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0297ac5b-2480-41da-ab70-392bfd3a9586 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:24:08.621066 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:24:08.663807 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3c843d-3c87-4224-9177-9b8355a005cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:24:08.664464 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3c843d-3c87-4224-9177-9b8355a005cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:24:08.664749 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3c843d-3c87-4224-9177-9b8355a005cf admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:24:08.665027 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3c843d-3c87-4224-9177-9b8355a005cf admin admin] PATH_INFO: `/v3/users/fe25209f8e2249d69d0ed28552826519` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:24:08.666220 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3c843d-3c87-4224-9177-9b8355a005cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe25209f8e2249d69d0ed28552826519)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:24:08.668911 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3c843d-3c87-4224-9177-9b8355a005cf admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:24:08.713156 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-6d3c843d-3c87-4224-9177-9b8355a005cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe25209f8e2249d69d0ed28552826519'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:24:08.720587 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-6d3c843d-3c87-4224-9177-9b8355a005cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe25209f8e2249d69d0ed28552826519'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:24:08.727483 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-6d3c843d-3c87-4224-9177-9b8355a005cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe25209f8e2249d69d0ed28552826519'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:24:08.728115 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-6d3c843d-3c87-4224-9177-9b8355a005cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe25209f8e2249d69d0ed28552826519'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:24:08.728728 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-6d3c843d-3c87-4224-9177-9b8355a005cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe25209f8e2249d69d0ed28552826519'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:24:08.745517 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1333/2655] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 14:24:08 2022] DELETE /identity/v3/users/fe25209f8e2249d69d0ed28552826519 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:24:08.925065 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6303437a-9d2a-4ec0-b796-784b4e9f2c33 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:24:08.927307 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-6303437a-9d2a-4ec0-b796-784b4e9f2c33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:24:08.941140 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6303437a-9d2a-4ec0-b796-784b4e9f2c33 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:24:08.967929 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:24:09.026551 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4cd84e-09c9-48f5-b4c2-30f226fea318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:24:09.027727 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4cd84e-09c9-48f5-b4c2-30f226fea318 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:24:09.028004 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4cd84e-09c9-48f5-b4c2-30f226fea318 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:24:09.028276 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4cd84e-09c9-48f5-b4c2-30f226fea318 admin admin] PATH_INFO: `/v3/projects/891db561dc1b424c900eae1fd95f5afb` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:24:09.029451 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4cd84e-09c9-48f5-b4c2-30f226fea318 admin admin] RBAC: Authorizing `identity:delete_project(project_id=891db561dc1b424c900eae1fd95f5afb)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:24:09.031407 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4cd84e-09c9-48f5-b4c2-30f226fea318 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:24:09.079357 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-7d4cd84e-09c9-48f5-b4c2-30f226fea318 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '891db561dc1b424c900eae1fd95f5afb', 'request_id': 'req-7d4cd84e-09c9-48f5-b4c2-30f226fea318'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:24:09.085986 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-7d4cd84e-09c9-48f5-b4c2-30f226fea318 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '891db561dc1b424c900eae1fd95f5afb', 'request_id': 'req-7d4cd84e-09c9-48f5-b4c2-30f226fea318'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:24:09.091511 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-7d4cd84e-09c9-48f5-b4c2-30f226fea318 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '891db561dc1b424c900eae1fd95f5afb', 'request_id': 'req-7d4cd84e-09c9-48f5-b4c2-30f226fea318'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:24:09.104700 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-7d4cd84e-09c9-48f5-b4c2-30f226fea318 admin admin] The token cache is being invalidate because project 891db561dc1b424c900eae1fd95f5afb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:24:09.104995 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-7d4cd84e-09c9-48f5-b4c2-30f226fea318 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:24:09.105668 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-7d4cd84e-09c9-48f5-b4c2-30f226fea318 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:24:09.107216 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1323/2656] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 14:24:08 2022] DELETE /identity/v3/projects/891db561dc1b424c900eae1fd95f5afb => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:24:14.708412 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3c843d-3c87-4224-9177-9b8355a005cf admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:24:14.709938 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-6d3c843d-3c87-4224-9177-9b8355a005cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:24:14.773791 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3c843d-3c87-4224-9177-9b8355a005cf admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:24:14.830449 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0724d8e0-16d6-4745-8d4a-e5da8d7f4322 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:24:14.831166 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0724d8e0-16d6-4745-8d4a-e5da8d7f4322 service neutron] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:24:14.831470 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0724d8e0-16d6-4745-8d4a-e5da8d7f4322 service neutron] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:24:14.831776 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0724d8e0-16d6-4745-8d4a-e5da8d7f4322 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:24:14.833545 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-0724d8e0-16d6-4745-8d4a-e5da8d7f4322 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:24:14.847949 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0724d8e0-16d6-4745-8d4a-e5da8d7f4322 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:24:14.849597 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0724d8e0-16d6-4745-8d4a-e5da8d7f4322 service neutron] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:24:14.858094 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1334/2657] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:24:14 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:24:14.942564 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4cd84e-09c9-48f5-b4c2-30f226fea318 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:24:14.943968 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-7d4cd84e-09c9-48f5-b4c2-30f226fea318 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:24:14.956501 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4cd84e-09c9-48f5-b4c2-30f226fea318 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:24:14.960021 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc07c8c-4769-4d18-91c4-0ff5f0fd6368 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:24:14.960625 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc07c8c-4769-4d18-91c4-0ff5f0fd6368 service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:24:14.960906 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc07c8c-4769-4d18-91c4-0ff5f0fd6368 service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:24:14.961175 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc07c8c-4769-4d18-91c4-0ff5f0fd6368 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:24:14.962908 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-3bc07c8c-4769-4d18-91c4-0ff5f0fd6368 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:24:15.033109 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc07c8c-4769-4d18-91c4-0ff5f0fd6368 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:24:15.034570 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc07c8c-4769-4d18-91c4-0ff5f0fd6368 service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:24:15.046414 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1324/2658] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:24:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:24:24.853848 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:24:25.040730 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=76,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:24:44.388467 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0724d8e0-16d6-4745-8d4a-e5da8d7f4322 service neutron] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:24:44.397573 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0724d8e0-16d6-4745-8d4a-e5da8d7f4322 service neutron] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:24:44.401296 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238c373b-bd41-4ab1-92a5-cce8490c855a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:24:44.401943 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c373b-bd41-4ab1-92a5-cce8490c855a service neutron] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:24:44.402230 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c373b-bd41-4ab1-92a5-cce8490c855a service neutron] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:24:44.403136 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c373b-bd41-4ab1-92a5-cce8490c855a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:24:44.405009 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-238c373b-bd41-4ab1-92a5-cce8490c855a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:24:44.474036 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238c373b-bd41-4ab1-92a5-cce8490c855a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:24:44.475392 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238c373b-bd41-4ab1-92a5-cce8490c855a service neutron] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:24:44.485440 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1335/2659] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:24:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:24:47.860863 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc07c8c-4769-4d18-91c4-0ff5f0fd6368 service neutron] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:24:47.862388 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-3bc07c8c-4769-4d18-91c4-0ff5f0fd6368 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:24:47.876684 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc07c8c-4769-4d18-91c4-0ff5f0fd6368 service neutron] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:24:47.881326 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffce3e5a-25c5-41a0-b48e-aaa416d3e9ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:24:47.881995 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffce3e5a-25c5-41a0-b48e-aaa416d3e9ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:24:47.882277 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffce3e5a-25c5-41a0-b48e-aaa416d3e9ba admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:24:47.882552 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffce3e5a-25c5-41a0-b48e-aaa416d3e9ba admin admin] PATH_INFO: `/v3/users/bd9ee1f0f2354f048dd6e809121fa0aa` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:24:47.883719 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffce3e5a-25c5-41a0-b48e-aaa416d3e9ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd9ee1f0f2354f048dd6e809121fa0aa)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:24:47.885916 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffce3e5a-25c5-41a0-b48e-aaa416d3e9ba admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:24:47.942831 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-ffce3e5a-25c5-41a0-b48e-aaa416d3e9ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd9ee1f0f2354f048dd6e809121fa0aa'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:24:47.943790 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-ffce3e5a-25c5-41a0-b48e-aaa416d3e9ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd9ee1f0f2354f048dd6e809121fa0aa'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:24:47.954175 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-ffce3e5a-25c5-41a0-b48e-aaa416d3e9ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd9ee1f0f2354f048dd6e809121fa0aa'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:24:47.955106 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-ffce3e5a-25c5-41a0-b48e-aaa416d3e9ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd9ee1f0f2354f048dd6e809121fa0aa'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:24:47.973661 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-ffce3e5a-25c5-41a0-b48e-aaa416d3e9ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd9ee1f0f2354f048dd6e809121fa0aa'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:24:47.989876 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1325/2660] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 14:24:47 2022] DELETE /identity/v3/users/bd9ee1f0f2354f048dd6e809121fa0aa => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:24:48.166568 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238c373b-bd41-4ab1-92a5-cce8490c855a service neutron] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:24:48.177810 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238c373b-bd41-4ab1-92a5-cce8490c855a service neutron] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:24:48.245232 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e60e44-ab10-4558-a9c0-e5dcc3401886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:24:48.246448 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e60e44-ab10-4558-a9c0-e5dcc3401886 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:24:48.246805 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e60e44-ab10-4558-a9c0-e5dcc3401886 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:24:48.247000 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e60e44-ab10-4558-a9c0-e5dcc3401886 admin admin] PATH_INFO: `/v3/projects/231dc10dd04c4bd39a971979ef1cb7a4` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:24:48.248221 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e60e44-ab10-4558-a9c0-e5dcc3401886 admin admin] RBAC: Authorizing `identity:delete_project(project_id=231dc10dd04c4bd39a971979ef1cb7a4)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:24:48.250155 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e60e44-ab10-4558-a9c0-e5dcc3401886 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:24:48.305536 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-08e60e44-ab10-4558-a9c0-e5dcc3401886 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '231dc10dd04c4bd39a971979ef1cb7a4', 'request_id': 'req-08e60e44-ab10-4558-a9c0-e5dcc3401886'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:24:48.314166 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-08e60e44-ab10-4558-a9c0-e5dcc3401886 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '231dc10dd04c4bd39a971979ef1cb7a4', 'request_id': 'req-08e60e44-ab10-4558-a9c0-e5dcc3401886'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:24:48.320089 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-08e60e44-ab10-4558-a9c0-e5dcc3401886 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '231dc10dd04c4bd39a971979ef1cb7a4', 'request_id': 'req-08e60e44-ab10-4558-a9c0-e5dcc3401886'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:24:48.332508 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-08e60e44-ab10-4558-a9c0-e5dcc3401886 admin admin] The token cache is being invalidate because project 231dc10dd04c4bd39a971979ef1cb7a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86933) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:24:48.332845 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-08e60e44-ab10-4558-a9c0-e5dcc3401886 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:24:48.333546 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-08e60e44-ab10-4558-a9c0-e5dcc3401886 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:24:48.335198 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1336/2661] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 14:24:48 2022] DELETE /identity/v3/projects/231dc10dd04c4bd39a971979ef1cb7a4 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:24:48.341585 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058a11ab-e7ff-4f48-b4e1-3490ce4395b7 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:24:48.341877 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058a11ab-e7ff-4f48-b4e1-3490ce4395b7 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:24:48.342153 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058a11ab-e7ff-4f48-b4e1-3490ce4395b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:24:48.362499 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-058a11ab-e7ff-4f48-b4e1-3490ce4395b7 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:24:48.363721 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-058a11ab-e7ff-4f48-b4e1-3490ce4395b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:24:48.426038 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1326/2662] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:24:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:24:48.434569 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38141f06-e31d-43d9-8141-636d2c484c1f None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:24:48.434859 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38141f06-e31d-43d9-8141-636d2c484c1f None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:24:48.435175 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38141f06-e31d-43d9-8141-636d2c484c1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:24:48.459239 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-38141f06-e31d-43d9-8141-636d2c484c1f None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:24:48.459467 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-38141f06-e31d-43d9-8141-636d2c484c1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:24:48.467410 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1337/2663] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:24:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:24:48.472010 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058a11ab-e7ff-4f48-b4e1-3490ce4395b7 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:24:48.482276 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058a11ab-e7ff-4f48-b4e1-3490ce4395b7 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:24:48.489191 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f458dc-259c-4ca4-be94-a8040562a047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:24:48.489896 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f458dc-259c-4ca4-be94-a8040562a047 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:24:48.490179 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f458dc-259c-4ca4-be94-a8040562a047 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:24:48.490486 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f458dc-259c-4ca4-be94-a8040562a047 admin admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:24:48.491235 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f458dc-259c-4ca4-be94-a8040562a047 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:24:48.491549 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f458dc-259c-4ca4-be94-a8040562a047 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:24:48.493274 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f458dc-259c-4ca4-be94-a8040562a047 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:24:48.506288 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1327/2664] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Jul 26 14:24:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:24:48.511271 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38141f06-e31d-43d9-8141-636d2c484c1f None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:24:48.521977 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38141f06-e31d-43d9-8141-636d2c484c1f None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:24:48.526842 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da275b9-cc03-47d8-a12d-cf800952997b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:24:48.527530 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da275b9-cc03-47d8-a12d-cf800952997b admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:24:48.527804 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da275b9-cc03-47d8-a12d-cf800952997b admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:24:48.528075 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da275b9-cc03-47d8-a12d-cf800952997b admin admin] PATH_INFO: `/v3/projects` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:24:48.528911 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da275b9-cc03-47d8-a12d-cf800952997b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:24:48.530834 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da275b9-cc03-47d8-a12d-cf800952997b admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:24:48.547280 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:24:48.562270 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1338/2665] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 14:24:48 2022] POST /identity/v3/projects => generated 365 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:24:48.567382 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f458dc-259c-4ca4-be94-a8040562a047 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:24:48.578486 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f458dc-259c-4ca4-be94-a8040562a047 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:24:48.634085 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39014964-8ff1-4089-9145-65d9dd230ff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:24:48.634698 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39014964-8ff1-4089-9145-65d9dd230ff1 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:24:48.634980 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39014964-8ff1-4089-9145-65d9dd230ff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:24:48.635250 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39014964-8ff1-4089-9145-65d9dd230ff1 admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:24:48.635912 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39014964-8ff1-4089-9145-65d9dd230ff1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:24:48.638481 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39014964-8ff1-4089-9145-65d9dd230ff1 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:24:48.646376 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:24:48.661506 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1328/2666] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 14:24:48 2022] POST /identity/v3/users => generated 283 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:24:48.665354 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da275b9-cc03-47d8-a12d-cf800952997b admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:24:48.676723 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da275b9-cc03-47d8-a12d-cf800952997b admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:24:48.681350 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfd129f-00b7-45b5-834c-1f5cd9f7e665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:24:48.681997 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfd129f-00b7-45b5-834c-1f5cd9f7e665 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:24:48.682281 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfd129f-00b7-45b5-834c-1f5cd9f7e665 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:24:48.682556 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfd129f-00b7-45b5-834c-1f5cd9f7e665 admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:24:48.683282 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfd129f-00b7-45b5-834c-1f5cd9f7e665 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:24:48.685349 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfd129f-00b7-45b5-834c-1f5cd9f7e665 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:24:48.695445 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1339/2667] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:24:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:24:48.700102 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39014964-8ff1-4089-9145-65d9dd230ff1 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:24:48.710480 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39014964-8ff1-4089-9145-65d9dd230ff1 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:24:48.715319 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0679012a-4076-4896-ae5c-77e0cc40108c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:24:48.715936 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0679012a-4076-4896-ae5c-77e0cc40108c admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:24:48.716218 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0679012a-4076-4896-ae5c-77e0cc40108c admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:24:48.716493 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0679012a-4076-4896-ae5c-77e0cc40108c admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:24:48.717191 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0679012a-4076-4896-ae5c-77e0cc40108c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:24:48.718837 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0679012a-4076-4896-ae5c-77e0cc40108c admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:24:48.730072 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1329/2668] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:24:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:24:48.734459 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfd129f-00b7-45b5-834c-1f5cd9f7e665 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:24:48.744674 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfd129f-00b7-45b5-834c-1f5cd9f7e665 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:24:48.749951 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635bc3a4-e896-435d-b5c6-7b50aa26b0db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:24:48.750621 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635bc3a4-e896-435d-b5c6-7b50aa26b0db admin admin] REQUEST_METHOD: `PUT` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:24:48.750912 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635bc3a4-e896-435d-b5c6-7b50aa26b0db admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:24:48.751330 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635bc3a4-e896-435d-b5c6-7b50aa26b0db admin admin] PATH_INFO: `/v3/projects/928490a7b1274101a15c3ebbff3efd17/users/dbf6dd19c5794ef58e88c3b396431536/roles/759f389b221049c4b01faf3b984cd8d7` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:24:48.767701 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635bc3a4-e896-435d-b5c6-7b50aa26b0db admin admin] RBAC: Authorizing `identity:create_grant(project_id=928490a7b1274101a15c3ebbff3efd17, user_id=dbf6dd19c5794ef58e88c3b396431536, role_id=759f389b221049c4b01faf3b984cd8d7)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:24:48.770512 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635bc3a4-e896-435d-b5c6-7b50aa26b0db admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:24:48.795412 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1340/2669] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 14:24:48 2022] PUT /identity/v3/projects/928490a7b1274101a15c3ebbff3efd17/users/dbf6dd19c5794ef58e88c3b396431536/roles/759f389b221049c4b01faf3b984cd8d7 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:24:48.807154 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f963decf-4d84-4049-aae1-cade425d4d03 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:24:48.807441 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f963decf-4d84-4049-aae1-cade425d4d03 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:24:48.807718 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f963decf-4d84-4049-aae1-cade425d4d03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:24:48.840244 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-f963decf-4d84-4049-aae1-cade425d4d03 None None] MFA Rules not processed for user `dbf6dd19c5794ef58e88c3b396431536`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:24:48.841517 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-f963decf-4d84-4049-aae1-cade425d4d03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:24:48.920365 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1330/2670] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:24:48 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:24:48.934237 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635bc3a4-e896-435d-b5c6-7b50aa26b0db admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:24:48.936673 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-635bc3a4-e896-435d-b5c6-7b50aa26b0db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:24:48.999628 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635bc3a4-e896-435d-b5c6-7b50aa26b0db admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:24:49.003691 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c40e7c4-81f1-420c-806e-6081dc3bc167 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:24:49.004309 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c40e7c4-81f1-420c-806e-6081dc3bc167 service cinder] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:24:49.004591 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c40e7c4-81f1-420c-806e-6081dc3bc167 service cinder] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:24:49.004869 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c40e7c4-81f1-420c-806e-6081dc3bc167 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:24:49.013931 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c40e7c4-81f1-420c-806e-6081dc3bc167 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:24:49.015357 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c40e7c4-81f1-420c-806e-6081dc3bc167 service cinder] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:24:49.025543 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1341/2671] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:24:48 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:24:57.523731 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f963decf-4d84-4049-aae1-cade425d4d03 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:24:57.596866 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f963decf-4d84-4049-aae1-cade425d4d03 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:24:57.601442 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62201394-78cd-4589-8dbe-11eb88c1f4b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:24:57.602139 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62201394-78cd-4589-8dbe-11eb88c1f4b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:24:57.602411 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62201394-78cd-4589-8dbe-11eb88c1f4b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:24:57.602685 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62201394-78cd-4589-8dbe-11eb88c1f4b9 admin admin] PATH_INFO: `/v3/users/dbf6dd19c5794ef58e88c3b396431536` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:24:57.603842 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62201394-78cd-4589-8dbe-11eb88c1f4b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbf6dd19c5794ef58e88c3b396431536)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:24:57.606396 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62201394-78cd-4589-8dbe-11eb88c1f4b9 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:24:57.654756 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-62201394-78cd-4589-8dbe-11eb88c1f4b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbf6dd19c5794ef58e88c3b396431536'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:24:57.655410 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-62201394-78cd-4589-8dbe-11eb88c1f4b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbf6dd19c5794ef58e88c3b396431536'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:24:57.662653 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-62201394-78cd-4589-8dbe-11eb88c1f4b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbf6dd19c5794ef58e88c3b396431536'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:24:57.663289 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-62201394-78cd-4589-8dbe-11eb88c1f4b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbf6dd19c5794ef58e88c3b396431536'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:24:57.676356 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:INSERT=3,keystone:SELECT=89,keystone:DELETE=8 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:24:57.679579 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-62201394-78cd-4589-8dbe-11eb88c1f4b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbf6dd19c5794ef58e88c3b396431536'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:24:57.694505 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1331/2672] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 14:24:57 2022] DELETE /identity/v3/users/dbf6dd19c5794ef58e88c3b396431536 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:24:57.700277 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29783aec-efb4-440b-9b24-5afa7ba88d74 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:24:57.700782 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29783aec-efb4-440b-9b24-5afa7ba88d74 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:24:57.700875 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29783aec-efb4-440b-9b24-5afa7ba88d74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:24:57.720018 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-29783aec-efb4-440b-9b24-5afa7ba88d74 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:24:57.721287 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-29783aec-efb4-440b-9b24-5afa7ba88d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:24:57.796903 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1342/2673] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:24:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:24:57.804574 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62201394-78cd-4589-8dbe-11eb88c1f4b9 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:24:57.807224 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-62201394-78cd-4589-8dbe-11eb88c1f4b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:24:57.874559 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62201394-78cd-4589-8dbe-11eb88c1f4b9 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:24:57.879639 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad869c39-8089-4ee3-ad6d-34875236b417 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:24:57.880255 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad869c39-8089-4ee3-ad6d-34875236b417 service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:24:57.880435 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad869c39-8089-4ee3-ad6d-34875236b417 service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:24:57.880713 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad869c39-8089-4ee3-ad6d-34875236b417 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:24:57.890551 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad869c39-8089-4ee3-ad6d-34875236b417 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:24:57.891951 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad869c39-8089-4ee3-ad6d-34875236b417 service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:24:57.901546 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1332/2674] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:24:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:24:58.187026 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29783aec-efb4-440b-9b24-5afa7ba88d74 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:24:58.188596 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-29783aec-efb4-440b-9b24-5afa7ba88d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:24:58.203381 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29783aec-efb4-440b-9b24-5afa7ba88d74 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:24:58.212090 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7197a74-f9e4-4061-b462-9c41295166b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:24:58.212768 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7197a74-f9e4-4061-b462-9c41295166b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:24:58.213042 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7197a74-f9e4-4061-b462-9c41295166b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:24:58.213317 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7197a74-f9e4-4061-b462-9c41295166b9 admin admin] PATH_INFO: `/v3/projects/928490a7b1274101a15c3ebbff3efd17` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:24:58.215872 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7197a74-f9e4-4061-b462-9c41295166b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=928490a7b1274101a15c3ebbff3efd17)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:24:58.217932 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7197a74-f9e4-4061-b462-9c41295166b9 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:24:58.268878 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-a7197a74-f9e4-4061-b462-9c41295166b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '928490a7b1274101a15c3ebbff3efd17', 'request_id': 'req-a7197a74-f9e4-4061-b462-9c41295166b9'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:24:58.276327 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-a7197a74-f9e4-4061-b462-9c41295166b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '928490a7b1274101a15c3ebbff3efd17', 'request_id': 'req-a7197a74-f9e4-4061-b462-9c41295166b9'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:24:58.278929 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:24:58.283561 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-a7197a74-f9e4-4061-b462-9c41295166b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '928490a7b1274101a15c3ebbff3efd17', 'request_id': 'req-a7197a74-f9e4-4061-b462-9c41295166b9'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:24:58.296633 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-a7197a74-f9e4-4061-b462-9c41295166b9 admin admin] The token cache is being invalidate because project 928490a7b1274101a15c3ebbff3efd17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86933) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:24:58.296829 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-a7197a74-f9e4-4061-b462-9c41295166b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:24:58.297502 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-a7197a74-f9e4-4061-b462-9c41295166b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:24:58.299019 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1343/2675] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 14:24:58 2022] DELETE /identity/v3/projects/928490a7b1274101a15c3ebbff3efd17 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:24:58.305627 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19aebd0-d25c-4b9a-ad15-ed7cb0364409 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:24:58.305920 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19aebd0-d25c-4b9a-ad15-ed7cb0364409 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:24:58.306194 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19aebd0-d25c-4b9a-ad15-ed7cb0364409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:24:58.326086 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-a19aebd0-d25c-4b9a-ad15-ed7cb0364409 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:24:58.327306 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-a19aebd0-d25c-4b9a-ad15-ed7cb0364409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:24:58.390215 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1333/2676] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:24:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:24:58.398267 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90a32a3-16bf-4cd8-b24e-71e3b459dd27 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:24:58.398556 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90a32a3-16bf-4cd8-b24e-71e3b459dd27 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:24:58.398841 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90a32a3-16bf-4cd8-b24e-71e3b459dd27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:24:58.421954 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-d90a32a3-16bf-4cd8-b24e-71e3b459dd27 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:24:58.423200 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-d90a32a3-16bf-4cd8-b24e-71e3b459dd27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:24:58.432201 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1344/2677] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:24:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:24:58.437067 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19aebd0-d25c-4b9a-ad15-ed7cb0364409 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:24:58.447168 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19aebd0-d25c-4b9a-ad15-ed7cb0364409 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:24:58.452263 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5476e2-a65f-4762-beef-460e1478b1f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:24:58.452945 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5476e2-a65f-4762-beef-460e1478b1f3 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:24:58.453262 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5476e2-a65f-4762-beef-460e1478b1f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:24:58.453652 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5476e2-a65f-4762-beef-460e1478b1f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:24:58.454192 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5476e2-a65f-4762-beef-460e1478b1f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:24:58.454496 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5476e2-a65f-4762-beef-460e1478b1f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:24:58.456102 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5476e2-a65f-4762-beef-460e1478b1f3 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:24:58.474032 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1334/2678] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Jul 26 14:24:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:24:58.478773 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90a32a3-16bf-4cd8-b24e-71e3b459dd27 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:24:58.489299 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90a32a3-16bf-4cd8-b24e-71e3b459dd27 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:24:58.494106 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe61e86e-441c-4a78-b7b5-884e1a9c9e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:24:58.494735 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe61e86e-441c-4a78-b7b5-884e1a9c9e6f admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:24:58.495020 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe61e86e-441c-4a78-b7b5-884e1a9c9e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:24:58.495293 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe61e86e-441c-4a78-b7b5-884e1a9c9e6f admin admin] PATH_INFO: `/v3/projects` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:24:58.496008 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe61e86e-441c-4a78-b7b5-884e1a9c9e6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:24:58.497931 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe61e86e-441c-4a78-b7b5-884e1a9c9e6f admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:24:58.529788 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1345/2679] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 14:24:58 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 14:24:58.534047 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5476e2-a65f-4762-beef-460e1478b1f3 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:24:58.544566 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5476e2-a65f-4762-beef-460e1478b1f3 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:24:58.608136 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0cb2b0-48be-4408-af5e-0e4a1bf3f170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:24:58.608751 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0cb2b0-48be-4408-af5e-0e4a1bf3f170 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:24:58.609031 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0cb2b0-48be-4408-af5e-0e4a1bf3f170 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:24:58.609332 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0cb2b0-48be-4408-af5e-0e4a1bf3f170 admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:24:58.610143 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0cb2b0-48be-4408-af5e-0e4a1bf3f170 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:24:58.612069 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0cb2b0-48be-4408-af5e-0e4a1bf3f170 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:24:58.633836 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1335/2680] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 14:24:58 2022] POST /identity/v3/users => generated 288 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:24:58.638777 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe61e86e-441c-4a78-b7b5-884e1a9c9e6f admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:24:58.649666 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe61e86e-441c-4a78-b7b5-884e1a9c9e6f admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:24:58.655239 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cd9a9e-fb41-442e-baa8-cac534c422ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:24:58.655865 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cd9a9e-fb41-442e-baa8-cac534c422ff admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:24:58.656147 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cd9a9e-fb41-442e-baa8-cac534c422ff admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:24:58.656424 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cd9a9e-fb41-442e-baa8-cac534c422ff admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:24:58.657126 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cd9a9e-fb41-442e-baa8-cac534c422ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:24:58.658776 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cd9a9e-fb41-442e-baa8-cac534c422ff admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:24:58.673134 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1346/2681] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:24:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:24:58.677588 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0cb2b0-48be-4408-af5e-0e4a1bf3f170 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:24:58.688540 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0cb2b0-48be-4408-af5e-0e4a1bf3f170 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:24:58.693592 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8bba60-fd4b-47c3-9d3b-28e316bb99b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:24:58.694237 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8bba60-fd4b-47c3-9d3b-28e316bb99b9 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:24:58.694520 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8bba60-fd4b-47c3-9d3b-28e316bb99b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:24:58.694799 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8bba60-fd4b-47c3-9d3b-28e316bb99b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:24:58.695625 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8bba60-fd4b-47c3-9d3b-28e316bb99b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:24:58.697231 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8bba60-fd4b-47c3-9d3b-28e316bb99b9 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:24:58.699616 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:24:58.712252 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1336/2682] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:24:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:24:58.717162 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cd9a9e-fb41-442e-baa8-cac534c422ff admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:24:58.727955 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cd9a9e-fb41-442e-baa8-cac534c422ff admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:24:58.732893 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf6544e-6d98-4c98-8520-513efbf2830e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:24:58.733618 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf6544e-6d98-4c98-8520-513efbf2830e admin admin] REQUEST_METHOD: `PUT` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:24:58.733928 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf6544e-6d98-4c98-8520-513efbf2830e admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:24:58.734209 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf6544e-6d98-4c98-8520-513efbf2830e admin admin] PATH_INFO: `/v3/projects/24346146ead5434991fd61b35649ea30/users/4287913348fe433cbccf977e29412c9b/roles/759f389b221049c4b01faf3b984cd8d7` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:24:58.754164 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf6544e-6d98-4c98-8520-513efbf2830e admin admin] RBAC: Authorizing `identity:create_grant(project_id=24346146ead5434991fd61b35649ea30, user_id=4287913348fe433cbccf977e29412c9b, role_id=759f389b221049c4b01faf3b984cd8d7)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:24:58.756849 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf6544e-6d98-4c98-8520-513efbf2830e admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:24:58.775504 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1347/2683] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 14:24:58 2022] PUT /identity/v3/projects/24346146ead5434991fd61b35649ea30/users/4287913348fe433cbccf977e29412c9b/roles/759f389b221049c4b01faf3b984cd8d7 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:24:58.788105 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2df636-e549-4905-9cbd-a3711ee5b656 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:24:58.788395 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2df636-e549-4905-9cbd-a3711ee5b656 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:24:58.788673 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2df636-e549-4905-9cbd-a3711ee5b656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:24:58.822486 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-ca2df636-e549-4905-9cbd-a3711ee5b656 None None] MFA Rules not processed for user `4287913348fe433cbccf977e29412c9b`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:24:58.823744 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-ca2df636-e549-4905-9cbd-a3711ee5b656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:24:58.913502 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1337/2684] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:24:58 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:24:58.926722 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf6544e-6d98-4c98-8520-513efbf2830e admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:24:58.929299 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-7bf6544e-6d98-4c98-8520-513efbf2830e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:24:59.022115 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf6544e-6d98-4c98-8520-513efbf2830e admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:24:59.028086 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fe5dc6-3a79-49bf-888d-8429043c46d4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c69161cd520b4b29bf50f3bf2f1c033b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:24:59.028716 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fe5dc6-3a79-49bf-888d-8429043c46d4 service nova] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:24:59.028995 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fe5dc6-3a79-49bf-888d-8429043c46d4 service nova] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:24:59.029270 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fe5dc6-3a79-49bf-888d-8429043c46d4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:24:59.038220 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fe5dc6-3a79-49bf-888d-8429043c46d4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:24:59.039873 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fe5dc6-3a79-49bf-888d-8429043c46d4 service nova] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:24:59.049709 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1348/2685] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:24:58 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:24:59.277788 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2df636-e549-4905-9cbd-a3711ee5b656 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:24:59.350753 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2df636-e549-4905-9cbd-a3711ee5b656 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:24:59.357624 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6ad11c-de8c-4050-ab11-b4c5411b5b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:24:59.358633 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6ad11c-de8c-4050-ab11-b4c5411b5b08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:24:59.359089 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6ad11c-de8c-4050-ab11-b4c5411b5b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:24:59.359542 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6ad11c-de8c-4050-ab11-b4c5411b5b08 admin admin] PATH_INFO: `/v3/users/4287913348fe433cbccf977e29412c9b` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:24:59.361518 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6ad11c-de8c-4050-ab11-b4c5411b5b08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4287913348fe433cbccf977e29412c9b)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:24:59.364915 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6ad11c-de8c-4050-ab11-b4c5411b5b08 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:24:59.427258 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-5d6ad11c-de8c-4050-ab11-b4c5411b5b08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4287913348fe433cbccf977e29412c9b'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:24:59.427973 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-5d6ad11c-de8c-4050-ab11-b4c5411b5b08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4287913348fe433cbccf977e29412c9b'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:24:59.435980 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-5d6ad11c-de8c-4050-ab11-b4c5411b5b08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4287913348fe433cbccf977e29412c9b'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:24:59.436680 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-5d6ad11c-de8c-4050-ab11-b4c5411b5b08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4287913348fe433cbccf977e29412c9b'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:24:59.451224 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-5d6ad11c-de8c-4050-ab11-b4c5411b5b08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4287913348fe433cbccf977e29412c9b'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:24:59.465142 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1338/2686] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 14:24:59 2022] DELETE /identity/v3/users/4287913348fe433cbccf977e29412c9b => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:24:59.469872 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e93aaf-5f40-437e-83da-42608e76f9cb None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:24:59.470291 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e93aaf-5f40-437e-83da-42608e76f9cb None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:24:59.470878 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e93aaf-5f40-437e-83da-42608e76f9cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:24:59.495007 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-b4e93aaf-5f40-437e-83da-42608e76f9cb None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:24:59.496845 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-b4e93aaf-5f40-437e-83da-42608e76f9cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:24:59.555270 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1349/2687] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:24:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:24:59.565764 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6ad11c-de8c-4050-ab11-b4c5411b5b08 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:24:59.567166 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-5d6ad11c-de8c-4050-ab11-b4c5411b5b08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:24:59.573873 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:24:59.632817 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6ad11c-de8c-4050-ab11-b4c5411b5b08 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:24:59.636643 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fab30ed-6f69-4c16-bd42-0c876062d89a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:24:59.637317 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fab30ed-6f69-4c16-bd42-0c876062d89a service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:24:59.637632 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fab30ed-6f69-4c16-bd42-0c876062d89a service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:24:59.637910 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fab30ed-6f69-4c16-bd42-0c876062d89a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:24:59.647069 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fab30ed-6f69-4c16-bd42-0c876062d89a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:24:59.648513 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fab30ed-6f69-4c16-bd42-0c876062d89a service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:24:59.658482 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1339/2688] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:24:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:24:59.930651 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e93aaf-5f40-437e-83da-42608e76f9cb None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:24:59.932243 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-b4e93aaf-5f40-437e-83da-42608e76f9cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:24:59.946220 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e93aaf-5f40-437e-83da-42608e76f9cb None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:24:59.951613 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac99510-21a0-40ac-8a6f-173e3c91d927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:24:59.952824 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac99510-21a0-40ac-8a6f-173e3c91d927 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:24:59.953142 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac99510-21a0-40ac-8a6f-173e3c91d927 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:24:59.953568 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac99510-21a0-40ac-8a6f-173e3c91d927 admin admin] PATH_INFO: `/v3/projects/24346146ead5434991fd61b35649ea30` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:24:59.955064 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac99510-21a0-40ac-8a6f-173e3c91d927 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24346146ead5434991fd61b35649ea30)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:24:59.957244 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac99510-21a0-40ac-8a6f-173e3c91d927 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:24:59.959931 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=97,keystone:DELETE=1,keystone:INSERT=2 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:25:00.018038 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-5ac99510-21a0-40ac-8a6f-173e3c91d927 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24346146ead5434991fd61b35649ea30', 'request_id': 'req-5ac99510-21a0-40ac-8a6f-173e3c91d927'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:25:00.025444 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-5ac99510-21a0-40ac-8a6f-173e3c91d927 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24346146ead5434991fd61b35649ea30', 'request_id': 'req-5ac99510-21a0-40ac-8a6f-173e3c91d927'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:25:00.032421 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-5ac99510-21a0-40ac-8a6f-173e3c91d927 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24346146ead5434991fd61b35649ea30', 'request_id': 'req-5ac99510-21a0-40ac-8a6f-173e3c91d927'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:25:00.044960 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-5ac99510-21a0-40ac-8a6f-173e3c91d927 admin admin] The token cache is being invalidate because project 24346146ead5434991fd61b35649ea30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86933) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:25:00.045252 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-5ac99510-21a0-40ac-8a6f-173e3c91d927 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:25:00.045949 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-5ac99510-21a0-40ac-8a6f-173e3c91d927 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:25:00.047504 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1350/2689] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 14:24:59 2022] DELETE /identity/v3/projects/24346146ead5434991fd61b35649ea30 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:25:00.052426 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ce4d37-1495-4ff6-8c4c-265abaed85e5 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:00.052745 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ce4d37-1495-4ff6-8c4c-265abaed85e5 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:00.053028 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ce4d37-1495-4ff6-8c4c-265abaed85e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:00.072663 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-29ce4d37-1495-4ff6-8c4c-265abaed85e5 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:25:00.074417 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-29ce4d37-1495-4ff6-8c4c-265abaed85e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:25:00.134575 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1340/2690] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:25:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:25:00.143010 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c318e52e-3aa7-4f9f-bddd-9ec646dbaf26 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:00.143305 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c318e52e-3aa7-4f9f-bddd-9ec646dbaf26 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:00.143780 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c318e52e-3aa7-4f9f-bddd-9ec646dbaf26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:00.164032 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-c318e52e-3aa7-4f9f-bddd-9ec646dbaf26 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:25:00.165255 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-c318e52e-3aa7-4f9f-bddd-9ec646dbaf26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:25:00.173431 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1351/2691] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:25:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:25:00.177848 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ce4d37-1495-4ff6-8c4c-265abaed85e5 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:25:00.187647 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ce4d37-1495-4ff6-8c4c-265abaed85e5 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:25:00.192283 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4cce53-e2c7-4fd4-9fad-0ddd16add5f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:25:00.192893 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4cce53-e2c7-4fd4-9fad-0ddd16add5f0 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:00.193171 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4cce53-e2c7-4fd4-9fad-0ddd16add5f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:00.193444 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4cce53-e2c7-4fd4-9fad-0ddd16add5f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:00.194161 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4cce53-e2c7-4fd4-9fad-0ddd16add5f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:25:00.194466 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4cce53-e2c7-4fd4-9fad-0ddd16add5f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:25:00.196042 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4cce53-e2c7-4fd4-9fad-0ddd16add5f0 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:25:00.208671 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1341/2692] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Jul 26 14:25:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:25:00.213772 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c318e52e-3aa7-4f9f-bddd-9ec646dbaf26 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:25:00.225128 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c318e52e-3aa7-4f9f-bddd-9ec646dbaf26 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:25:00.229906 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08f959a-b5c9-4eb7-ad3f-d1fc28221181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:25:00.230552 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08f959a-b5c9-4eb7-ad3f-d1fc28221181 admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:00.230833 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08f959a-b5c9-4eb7-ad3f-d1fc28221181 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:00.231106 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08f959a-b5c9-4eb7-ad3f-d1fc28221181 admin admin] PATH_INFO: `/v3/projects` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:00.231769 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08f959a-b5c9-4eb7-ad3f-d1fc28221181 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:25:00.233682 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08f959a-b5c9-4eb7-ad3f-d1fc28221181 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:25:00.262894 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1352/2693] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 14:25:00 2022] POST /identity/v3/projects => generated 363 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:25:00.267543 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4cce53-e2c7-4fd4-9fad-0ddd16add5f0 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:25:00.279667 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4cce53-e2c7-4fd4-9fad-0ddd16add5f0 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:25:00.338946 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f527d6d5-7697-40ae-8c4c-1651fc0f6c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:25:00.339658 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f527d6d5-7697-40ae-8c4c-1651fc0f6c73 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:00.340579 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f527d6d5-7697-40ae-8c4c-1651fc0f6c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:00.340908 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f527d6d5-7697-40ae-8c4c-1651fc0f6c73 admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:00.341707 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f527d6d5-7697-40ae-8c4c-1651fc0f6c73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:25:00.344278 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f527d6d5-7697-40ae-8c4c-1651fc0f6c73 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:25:00.366185 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1342/2694] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 14:25:00 2022] POST /identity/v3/users => generated 282 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:25:00.370435 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08f959a-b5c9-4eb7-ad3f-d1fc28221181 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:25:00.381739 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08f959a-b5c9-4eb7-ad3f-d1fc28221181 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:25:00.387463 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29278f19-fcd0-4d38-bcae-c11c68955fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:25:00.388076 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29278f19-fcd0-4d38-bcae-c11c68955fc5 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:00.388355 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29278f19-fcd0-4d38-bcae-c11c68955fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:00.388627 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29278f19-fcd0-4d38-bcae-c11c68955fc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:00.389317 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29278f19-fcd0-4d38-bcae-c11c68955fc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:25:00.390945 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29278f19-fcd0-4d38-bcae-c11c68955fc5 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:25:00.401468 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1353/2695] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:25:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:25:00.405687 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f527d6d5-7697-40ae-8c4c-1651fc0f6c73 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:25:00.416015 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f527d6d5-7697-40ae-8c4c-1651fc0f6c73 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:25:00.420739 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6df5ccd-29c0-438a-9e06-03bbfdefbb4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:25:00.421358 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6df5ccd-29c0-438a-9e06-03bbfdefbb4b admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:00.421684 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6df5ccd-29c0-438a-9e06-03bbfdefbb4b admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:00.421960 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6df5ccd-29c0-438a-9e06-03bbfdefbb4b admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:00.422657 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6df5ccd-29c0-438a-9e06-03bbfdefbb4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:25:00.424373 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6df5ccd-29c0-438a-9e06-03bbfdefbb4b admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:25:00.436298 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1343/2696] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:25:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:25:00.440580 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29278f19-fcd0-4d38-bcae-c11c68955fc5 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:25:00.451356 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29278f19-fcd0-4d38-bcae-c11c68955fc5 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:25:00.456732 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0ac748-7ae7-4d30-b1e5-7e9259a31972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:25:00.457418 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0ac748-7ae7-4d30-b1e5-7e9259a31972 admin admin] REQUEST_METHOD: `PUT` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:00.457745 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0ac748-7ae7-4d30-b1e5-7e9259a31972 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:00.458024 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0ac748-7ae7-4d30-b1e5-7e9259a31972 admin admin] PATH_INFO: `/v3/projects/94ed0c5a842943fb98a24506badc88f2/users/859d2e6f43c44a22b0c3f6f85beee7e7/roles/759f389b221049c4b01faf3b984cd8d7` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:00.474078 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0ac748-7ae7-4d30-b1e5-7e9259a31972 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94ed0c5a842943fb98a24506badc88f2, user_id=859d2e6f43c44a22b0c3f6f85beee7e7, role_id=759f389b221049c4b01faf3b984cd8d7)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:25:00.476801 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0ac748-7ae7-4d30-b1e5-7e9259a31972 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:25:00.491425 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1354/2697] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 14:25:00 2022] PUT /identity/v3/projects/94ed0c5a842943fb98a24506badc88f2/users/859d2e6f43c44a22b0c3f6f85beee7e7/roles/759f389b221049c4b01faf3b984cd8d7 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:25:00.503406 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23343036-b27a-4a98-99bc-5fead94aebe7 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:00.503726 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23343036-b27a-4a98-99bc-5fead94aebe7 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:00.504156 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23343036-b27a-4a98-99bc-5fead94aebe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:00.531127 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:25:00.534593 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-23343036-b27a-4a98-99bc-5fead94aebe7 None None] MFA Rules not processed for user `859d2e6f43c44a22b0c3f6f85beee7e7`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:25:00.536503 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-23343036-b27a-4a98-99bc-5fead94aebe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:25:00.599865 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1344/2698] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:25:00 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:25:00.604494 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0ac748-7ae7-4d30-b1e5-7e9259a31972 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:25:00.667747 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0ac748-7ae7-4d30-b1e5-7e9259a31972 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:25:00.672934 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0facd16e-ac4f-482f-903d-0c8262a84b36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:25:00.673599 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0facd16e-ac4f-482f-903d-0c8262a84b36 admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:00.673881 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0facd16e-ac4f-482f-903d-0c8262a84b36 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:00.674164 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0facd16e-ac4f-482f-903d-0c8262a84b36 admin admin] PATH_INFO: `/v3/projects` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:00.674886 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0facd16e-ac4f-482f-903d-0c8262a84b36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:25:00.676758 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0facd16e-ac4f-482f-903d-0c8262a84b36 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:25:00.704875 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1355/2699] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 14:25:00 2022] POST /identity/v3/projects => generated 363 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:25:00.709243 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23343036-b27a-4a98-99bc-5fead94aebe7 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:25:00.720143 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23343036-b27a-4a98-99bc-5fead94aebe7 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:25:00.777301 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3319bbc4-9683-4ae7-ae83-ecb1ba8044d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:25:00.778039 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3319bbc4-9683-4ae7-ae83-ecb1ba8044d4 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:00.778285 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3319bbc4-9683-4ae7-ae83-ecb1ba8044d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:00.778557 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3319bbc4-9683-4ae7-ae83-ecb1ba8044d4 admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:00.779241 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3319bbc4-9683-4ae7-ae83-ecb1ba8044d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:25:00.781109 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3319bbc4-9683-4ae7-ae83-ecb1ba8044d4 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:25:00.803677 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1345/2700] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 14:25:00 2022] POST /identity/v3/users => generated 288 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:25:00.807730 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0facd16e-ac4f-482f-903d-0c8262a84b36 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:25:00.819351 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0facd16e-ac4f-482f-903d-0c8262a84b36 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:25:00.825274 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d615a4-61be-474d-9729-c536eb1334b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:25:00.825951 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d615a4-61be-474d-9729-c536eb1334b8 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:00.826236 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d615a4-61be-474d-9729-c536eb1334b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:00.826514 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d615a4-61be-474d-9729-c536eb1334b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:00.827211 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d615a4-61be-474d-9729-c536eb1334b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:25:00.828794 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d615a4-61be-474d-9729-c536eb1334b8 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:25:00.964503 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1356/2701] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:25:00 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:25:00.968675 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3319bbc4-9683-4ae7-ae83-ecb1ba8044d4 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:25:00.981462 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3319bbc4-9683-4ae7-ae83-ecb1ba8044d4 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:25:00.987043 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bffc31-8dcb-4c79-b6d3-203eb0b5b7cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:25:00.987815 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bffc31-8dcb-4c79-b6d3-203eb0b5b7cb admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:00.988142 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bffc31-8dcb-4c79-b6d3-203eb0b5b7cb admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:00.988447 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bffc31-8dcb-4c79-b6d3-203eb0b5b7cb admin admin] PATH_INFO: `/v3/projects/35e659d5c72144efad2fe00095f422e6/users/55fc46a9d899436a805ae95aa4964bc2/roles/6e0201a9a6484f7d8b112b1eafff70bd` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:01.007091 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bffc31-8dcb-4c79-b6d3-203eb0b5b7cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=35e659d5c72144efad2fe00095f422e6, user_id=55fc46a9d899436a805ae95aa4964bc2, role_id=6e0201a9a6484f7d8b112b1eafff70bd)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:25:01.010725 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bffc31-8dcb-4c79-b6d3-203eb0b5b7cb admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:25:01.034986 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1346/2702] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 14:25:00 2022] PUT /identity/v3/projects/35e659d5c72144efad2fe00095f422e6/users/55fc46a9d899436a805ae95aa4964bc2/roles/6e0201a9a6484f7d8b112b1eafff70bd => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:25:01.036971 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d615a4-61be-474d-9729-c536eb1334b8 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:25:01.042446 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:25:01.050176 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d615a4-61be-474d-9729-c536eb1334b8 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:25:01.110546 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74d9626-e789-48e0-8fc1-92aaa75c8afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:25:01.111265 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74d9626-e789-48e0-8fc1-92aaa75c8afd admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:01.111573 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74d9626-e789-48e0-8fc1-92aaa75c8afd admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:01.111815 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74d9626-e789-48e0-8fc1-92aaa75c8afd admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:01.112601 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74d9626-e789-48e0-8fc1-92aaa75c8afd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:25:01.114136 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74d9626-e789-48e0-8fc1-92aaa75c8afd admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:25:01.124468 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1357/2703] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:25:01 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:25:01.129055 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bffc31-8dcb-4c79-b6d3-203eb0b5b7cb admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:25:01.140471 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bffc31-8dcb-4c79-b6d3-203eb0b5b7cb admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:25:01.145694 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9e742c-2be2-4c72-aaa4-4ac5adf3b277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:25:01.146357 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e742c-2be2-4c72-aaa4-4ac5adf3b277 admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:01.146634 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e742c-2be2-4c72-aaa4-4ac5adf3b277 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:01.146925 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e742c-2be2-4c72-aaa4-4ac5adf3b277 admin admin] PATH_INFO: `/v3/projects/35e659d5c72144efad2fe00095f422e6/users/55fc46a9d899436a805ae95aa4964bc2/roles/6e0201a9a6484f7d8b112b1eafff70bd` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:01.148943 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9e742c-2be2-4c72-aaa4-4ac5adf3b277 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35e659d5c72144efad2fe00095f422e6, user_id=55fc46a9d899436a805ae95aa4964bc2, role_id=6e0201a9a6484f7d8b112b1eafff70bd)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:25:01.151623 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9e742c-2be2-4c72-aaa4-4ac5adf3b277 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:25:01.164414 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1347/2704] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 14:25:01 2022] PUT /identity/v3/projects/35e659d5c72144efad2fe00095f422e6/users/55fc46a9d899436a805ae95aa4964bc2/roles/6e0201a9a6484f7d8b112b1eafff70bd => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:25:01.177314 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2bd858-cdbe-42a2-b0a0-892aa1751ddf None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:01.177627 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2bd858-cdbe-42a2-b0a0-892aa1751ddf None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:01.177905 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2bd858-cdbe-42a2-b0a0-892aa1751ddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:01.205528 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-3e2bd858-cdbe-42a2-b0a0-892aa1751ddf None None] MFA Rules not processed for user `55fc46a9d899436a805ae95aa4964bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:25:01.206725 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-3e2bd858-cdbe-42a2-b0a0-892aa1751ddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:25:01.280065 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1358/2705] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:25:01 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:25:01.292323 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9e742c-2be2-4c72-aaa4-4ac5adf3b277 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:25:01.294880 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-6a9e742c-2be2-4c72-aaa4-4ac5adf3b277 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:25:01.368182 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9e742c-2be2-4c72-aaa4-4ac5adf3b277 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:25:01.371894 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c40c99-b082-46f8-8cd5-61a74a528f79 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:25:01.372552 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c40c99-b082-46f8-8cd5-61a74a528f79 service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:01.372831 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c40c99-b082-46f8-8cd5-61a74a528f79 service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:01.373103 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c40c99-b082-46f8-8cd5-61a74a528f79 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:01.382940 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c40c99-b082-46f8-8cd5-61a74a528f79 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:25:01.384689 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c40c99-b082-46f8-8cd5-61a74a528f79 service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:25:01.388862 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:25:01.397156 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1348/2706] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:25:01 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:25:01.485831 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2bd858-cdbe-42a2-b0a0-892aa1751ddf None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:25:01.495681 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2bd858-cdbe-42a2-b0a0-892aa1751ddf None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:25:01.499392 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acab366-ed58-4b29-b897-83c7488e05b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:25:01.500002 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acab366-ed58-4b29-b897-83c7488e05b7 service cinder] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:01.500286 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acab366-ed58-4b29-b897-83c7488e05b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:01.500561 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acab366-ed58-4b29-b897-83c7488e05b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:01.557098 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acab366-ed58-4b29-b897-83c7488e05b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:25:01.558490 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acab366-ed58-4b29-b897-83c7488e05b7 service cinder] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:25:01.567588 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1359/2707] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:25:01 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:25:11.397617 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:25:11.563715 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=90 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:25:11.911495 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c40c99-b082-46f8-8cd5-61a74a528f79 service cinder] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:25:11.980476 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c40c99-b082-46f8-8cd5-61a74a528f79 service cinder] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:25:11.986302 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2075f84f-e5c2-46e6-b3f2-5ff3a3f2b5bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:25:11.986951 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2075f84f-e5c2-46e6-b3f2-5ff3a3f2b5bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:11.987348 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2075f84f-e5c2-46e6-b3f2-5ff3a3f2b5bd admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:11.987626 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2075f84f-e5c2-46e6-b3f2-5ff3a3f2b5bd admin admin] PATH_INFO: `/v3/users/859d2e6f43c44a22b0c3f6f85beee7e7` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:11.988840 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2075f84f-e5c2-46e6-b3f2-5ff3a3f2b5bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=859d2e6f43c44a22b0c3f6f85beee7e7)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:25:11.990756 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2075f84f-e5c2-46e6-b3f2-5ff3a3f2b5bd admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:25:12.043754 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-2075f84f-e5c2-46e6-b3f2-5ff3a3f2b5bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '859d2e6f43c44a22b0c3f6f85beee7e7'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:25:12.044403 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-2075f84f-e5c2-46e6-b3f2-5ff3a3f2b5bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '859d2e6f43c44a22b0c3f6f85beee7e7'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:25:12.056359 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-2075f84f-e5c2-46e6-b3f2-5ff3a3f2b5bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '859d2e6f43c44a22b0c3f6f85beee7e7'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:25:12.057098 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-2075f84f-e5c2-46e6-b3f2-5ff3a3f2b5bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '859d2e6f43c44a22b0c3f6f85beee7e7'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:25:12.070649 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-2075f84f-e5c2-46e6-b3f2-5ff3a3f2b5bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '859d2e6f43c44a22b0c3f6f85beee7e7'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:25:12.085320 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1349/2708] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 14:25:11 2022] DELETE /identity/v3/users/859d2e6f43c44a22b0c3f6f85beee7e7 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:25:12.090868 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb0c6d3-c4b1-4f08-98a8-148df0dfa084 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:12.091267 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb0c6d3-c4b1-4f08-98a8-148df0dfa084 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:12.091647 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb0c6d3-c4b1-4f08-98a8-148df0dfa084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:12.110923 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-efb0c6d3-c4b1-4f08-98a8-148df0dfa084 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:25:12.112126 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-efb0c6d3-c4b1-4f08-98a8-148df0dfa084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:25:12.177142 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1360/2709] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:25:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:25:12.188921 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2075f84f-e5c2-46e6-b3f2-5ff3a3f2b5bd admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:25:12.190432 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-2075f84f-e5c2-46e6-b3f2-5ff3a3f2b5bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:25:12.249742 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2075f84f-e5c2-46e6-b3f2-5ff3a3f2b5bd admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:25:12.253305 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6035e8-ea2f-404f-963b-9d5a4ad3343e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:25:12.254159 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6035e8-ea2f-404f-963b-9d5a4ad3343e service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:12.254439 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6035e8-ea2f-404f-963b-9d5a4ad3343e service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:12.254712 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6035e8-ea2f-404f-963b-9d5a4ad3343e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:12.265137 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6035e8-ea2f-404f-963b-9d5a4ad3343e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:25:12.266628 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6035e8-ea2f-404f-963b-9d5a4ad3343e service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:25:12.276729 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1350/2710] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:25:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:25:12.583261 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb0c6d3-c4b1-4f08-98a8-148df0dfa084 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:25:12.585091 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-efb0c6d3-c4b1-4f08-98a8-148df0dfa084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:25:12.602360 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb0c6d3-c4b1-4f08-98a8-148df0dfa084 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:25:12.607661 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bd175c-018d-43b1-ad83-df6b040b12c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:25:12.608320 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bd175c-018d-43b1-ad83-df6b040b12c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:12.608600 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bd175c-018d-43b1-ad83-df6b040b12c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:12.608873 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bd175c-018d-43b1-ad83-df6b040b12c0 admin admin] PATH_INFO: `/v3/projects/94ed0c5a842943fb98a24506badc88f2` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:12.610060 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bd175c-018d-43b1-ad83-df6b040b12c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94ed0c5a842943fb98a24506badc88f2)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:25:12.611977 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bd175c-018d-43b1-ad83-df6b040b12c0 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:25:12.670183 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-f2bd175c-018d-43b1-ad83-df6b040b12c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94ed0c5a842943fb98a24506badc88f2', 'request_id': 'req-f2bd175c-018d-43b1-ad83-df6b040b12c0'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:25:12.679082 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-f2bd175c-018d-43b1-ad83-df6b040b12c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94ed0c5a842943fb98a24506badc88f2', 'request_id': 'req-f2bd175c-018d-43b1-ad83-df6b040b12c0'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:25:12.685652 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-f2bd175c-018d-43b1-ad83-df6b040b12c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94ed0c5a842943fb98a24506badc88f2', 'request_id': 'req-f2bd175c-018d-43b1-ad83-df6b040b12c0'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:25:12.698145 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-f2bd175c-018d-43b1-ad83-df6b040b12c0 admin admin] The token cache is being invalidate because project 94ed0c5a842943fb98a24506badc88f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86933) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:25:12.698444 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-f2bd175c-018d-43b1-ad83-df6b040b12c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:25:12.699160 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-f2bd175c-018d-43b1-ad83-df6b040b12c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:25:12.700718 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1361/2711] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 14:25:12 2022] DELETE /identity/v3/projects/94ed0c5a842943fb98a24506badc88f2 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:25:12.706478 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6035e8-ea2f-404f-963b-9d5a4ad3343e service neutron] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:25:12.716693 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6035e8-ea2f-404f-963b-9d5a4ad3343e service neutron] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:25:12.739944 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:25:12.784453 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af22e8c5-d76f-4dbc-b120-a633b6e6b0b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:25:12.785100 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af22e8c5-d76f-4dbc-b120-a633b6e6b0b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:12.785386 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af22e8c5-d76f-4dbc-b120-a633b6e6b0b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:12.785693 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af22e8c5-d76f-4dbc-b120-a633b6e6b0b0 admin admin] PATH_INFO: `/v3/users/55fc46a9d899436a805ae95aa4964bc2` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:12.786921 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af22e8c5-d76f-4dbc-b120-a633b6e6b0b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55fc46a9d899436a805ae95aa4964bc2)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:25:12.788878 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af22e8c5-d76f-4dbc-b120-a633b6e6b0b0 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:25:12.841056 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-af22e8c5-d76f-4dbc-b120-a633b6e6b0b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55fc46a9d899436a805ae95aa4964bc2'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:25:12.841961 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-af22e8c5-d76f-4dbc-b120-a633b6e6b0b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55fc46a9d899436a805ae95aa4964bc2'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:25:12.850095 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-af22e8c5-d76f-4dbc-b120-a633b6e6b0b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55fc46a9d899436a805ae95aa4964bc2'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:25:12.850787 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-af22e8c5-d76f-4dbc-b120-a633b6e6b0b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55fc46a9d899436a805ae95aa4964bc2'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:25:12.863828 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-af22e8c5-d76f-4dbc-b120-a633b6e6b0b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55fc46a9d899436a805ae95aa4964bc2'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:25:12.878471 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1351/2712] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 14:25:12 2022] DELETE /identity/v3/users/55fc46a9d899436a805ae95aa4964bc2 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:25:13.176430 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bd175c-018d-43b1-ad83-df6b040b12c0 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:25:13.178043 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-f2bd175c-018d-43b1-ad83-df6b040b12c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:25:13.202654 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bd175c-018d-43b1-ad83-df6b040b12c0 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:25:13.273291 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc5c2ce-c4a3-494b-9046-016742d2523c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:25:13.273942 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc5c2ce-c4a3-494b-9046-016742d2523c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:13.274173 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc5c2ce-c4a3-494b-9046-016742d2523c admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:13.274446 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc5c2ce-c4a3-494b-9046-016742d2523c admin admin] PATH_INFO: `/v3/projects/35e659d5c72144efad2fe00095f422e6` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:13.275642 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc5c2ce-c4a3-494b-9046-016742d2523c admin admin] RBAC: Authorizing `identity:delete_project(project_id=35e659d5c72144efad2fe00095f422e6)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:25:13.277602 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc5c2ce-c4a3-494b-9046-016742d2523c admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:25:13.314068 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:25:13.334402 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-ebc5c2ce-c4a3-494b-9046-016742d2523c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35e659d5c72144efad2fe00095f422e6', 'request_id': 'req-ebc5c2ce-c4a3-494b-9046-016742d2523c'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:25:13.342302 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-ebc5c2ce-c4a3-494b-9046-016742d2523c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35e659d5c72144efad2fe00095f422e6', 'request_id': 'req-ebc5c2ce-c4a3-494b-9046-016742d2523c'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:25:13.348054 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-ebc5c2ce-c4a3-494b-9046-016742d2523c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35e659d5c72144efad2fe00095f422e6', 'request_id': 'req-ebc5c2ce-c4a3-494b-9046-016742d2523c'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:25:13.361315 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-ebc5c2ce-c4a3-494b-9046-016742d2523c admin admin] The token cache is being invalidate because project 35e659d5c72144efad2fe00095f422e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86933) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:25:13.361629 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-ebc5c2ce-c4a3-494b-9046-016742d2523c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:25:13.362301 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-ebc5c2ce-c4a3-494b-9046-016742d2523c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:25:13.363834 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1362/2713] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 14:25:13 2022] DELETE /identity/v3/projects/35e659d5c72144efad2fe00095f422e6 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:25:13.368746 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b2a273-09b3-4b95-834d-513eb2083187 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:13.369050 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b2a273-09b3-4b95-834d-513eb2083187 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:13.369316 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b2a273-09b3-4b95-834d-513eb2083187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:13.391712 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-b4b2a273-09b3-4b95-834d-513eb2083187 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:25:13.392798 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-b4b2a273-09b3-4b95-834d-513eb2083187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:25:13.462484 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1352/2714] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:25:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:25:13.470819 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a87960-829f-4bfa-8fc7-d1fb85d62223 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:13.471101 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a87960-829f-4bfa-8fc7-d1fb85d62223 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:13.471389 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a87960-829f-4bfa-8fc7-d1fb85d62223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:13.490883 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-44a87960-829f-4bfa-8fc7-d1fb85d62223 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:25:13.492103 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-44a87960-829f-4bfa-8fc7-d1fb85d62223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:25:13.500724 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1363/2715] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:25:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:25:13.505220 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b2a273-09b3-4b95-834d-513eb2083187 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:25:13.515846 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b2a273-09b3-4b95-834d-513eb2083187 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:25:13.521461 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f06025-8478-4ad7-b04e-f77b86dedad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:25:13.522116 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f06025-8478-4ad7-b04e-f77b86dedad5 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:13.522435 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f06025-8478-4ad7-b04e-f77b86dedad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:13.522704 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f06025-8478-4ad7-b04e-f77b86dedad5 admin admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:13.523453 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f06025-8478-4ad7-b04e-f77b86dedad5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:25:13.523725 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f06025-8478-4ad7-b04e-f77b86dedad5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:25:13.525372 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f06025-8478-4ad7-b04e-f77b86dedad5 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:25:13.539056 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1353/2716] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Jul 26 14:25:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:25:13.543917 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a87960-829f-4bfa-8fc7-d1fb85d62223 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:25:13.553341 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a87960-829f-4bfa-8fc7-d1fb85d62223 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:25:13.558817 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fc1707-f991-4ee8-a066-561db638e4a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:25:13.559425 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fc1707-f991-4ee8-a066-561db638e4a9 admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:13.559703 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fc1707-f991-4ee8-a066-561db638e4a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:13.559970 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fc1707-f991-4ee8-a066-561db638e4a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:13.560640 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fc1707-f991-4ee8-a066-561db638e4a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:25:13.562645 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fc1707-f991-4ee8-a066-561db638e4a9 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:25:13.595691 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1364/2717] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 14:25:13 2022] POST /identity/v3/projects => generated 377 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:25:13.599917 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f06025-8478-4ad7-b04e-f77b86dedad5 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:25:13.610688 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f06025-8478-4ad7-b04e-f77b86dedad5 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:25:13.637368 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:25:13.672139 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d1e5d8-3056-4bcb-a9b2-2f753c436b82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:25:13.672755 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d1e5d8-3056-4bcb-a9b2-2f753c436b82 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:13.673041 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d1e5d8-3056-4bcb-a9b2-2f753c436b82 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:13.673316 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d1e5d8-3056-4bcb-a9b2-2f753c436b82 admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:13.674197 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d1e5d8-3056-4bcb-a9b2-2f753c436b82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:25:13.676105 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d1e5d8-3056-4bcb-a9b2-2f753c436b82 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:25:13.698344 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1354/2718] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 14:25:13 2022] POST /identity/v3/users => generated 289 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:25:13.702642 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fc1707-f991-4ee8-a066-561db638e4a9 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:25:13.713043 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fc1707-f991-4ee8-a066-561db638e4a9 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:25:13.717847 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d09183-4f83-47b9-89f3-ed01eba409ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:25:13.718497 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d09183-4f83-47b9-89f3-ed01eba409ab admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:13.718774 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d09183-4f83-47b9-89f3-ed01eba409ab admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:13.719045 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d09183-4f83-47b9-89f3-ed01eba409ab admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:13.719791 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d09183-4f83-47b9-89f3-ed01eba409ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:25:13.722743 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d09183-4f83-47b9-89f3-ed01eba409ab admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:25:13.742090 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1365/2719] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:25:13 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 14:25:13.747421 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d1e5d8-3056-4bcb-a9b2-2f753c436b82 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:25:13.757463 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d1e5d8-3056-4bcb-a9b2-2f753c436b82 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:25:13.762727 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d54d672-e612-4438-8cce-93e82486c861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:25:13.763344 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d54d672-e612-4438-8cce-93e82486c861 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:13.763624 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d54d672-e612-4438-8cce-93e82486c861 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:13.763897 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d54d672-e612-4438-8cce-93e82486c861 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:13.764590 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d54d672-e612-4438-8cce-93e82486c861 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:25:13.766460 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d54d672-e612-4438-8cce-93e82486c861 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:25:13.780917 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1355/2720] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:25:13 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:25:13.785392 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d09183-4f83-47b9-89f3-ed01eba409ab admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:25:13.795994 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d09183-4f83-47b9-89f3-ed01eba409ab admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:25:13.801745 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac27b81-d870-408d-aa5e-d00a0936fe44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:25:13.802441 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac27b81-d870-408d-aa5e-d00a0936fe44 admin admin] REQUEST_METHOD: `PUT` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:13.802719 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac27b81-d870-408d-aa5e-d00a0936fe44 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:13.802997 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac27b81-d870-408d-aa5e-d00a0936fe44 admin admin] PATH_INFO: `/v3/projects/699b3d1746c84b18a6f994a13c2c504f/users/42bc9208462947009c2e35b23e243f59/roles/759f389b221049c4b01faf3b984cd8d7` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:13.818630 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac27b81-d870-408d-aa5e-d00a0936fe44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=699b3d1746c84b18a6f994a13c2c504f, user_id=42bc9208462947009c2e35b23e243f59, role_id=759f389b221049c4b01faf3b984cd8d7)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:25:13.821230 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac27b81-d870-408d-aa5e-d00a0936fe44 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:25:13.834477 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1366/2721] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 14:25:13 2022] PUT /identity/v3/projects/699b3d1746c84b18a6f994a13c2c504f/users/42bc9208462947009c2e35b23e243f59/roles/759f389b221049c4b01faf3b984cd8d7 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:25:13.845898 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac353df-db84-432b-9816-3fac66674491 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:13.846187 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac353df-db84-432b-9816-3fac66674491 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:13.846464 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac353df-db84-432b-9816-3fac66674491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:13.876744 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-0ac353df-db84-432b-9816-3fac66674491 None None] MFA Rules not processed for user `42bc9208462947009c2e35b23e243f59`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:25:13.878120 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-0ac353df-db84-432b-9816-3fac66674491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:25:13.943974 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1356/2722] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:25:13 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:25:13.948526 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac27b81-d870-408d-aa5e-d00a0936fe44 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:25:14.013320 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac27b81-d870-408d-aa5e-d00a0936fe44 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:25:14.018257 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a232d2fe-cfd3-4f58-a319-b29f2d5083d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:25:14.018849 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a232d2fe-cfd3-4f58-a319-b29f2d5083d3 admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:14.019197 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a232d2fe-cfd3-4f58-a319-b29f2d5083d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:14.019451 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a232d2fe-cfd3-4f58-a319-b29f2d5083d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:14.020112 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a232d2fe-cfd3-4f58-a319-b29f2d5083d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:25:14.022043 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a232d2fe-cfd3-4f58-a319-b29f2d5083d3 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:25:14.050963 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1367/2723] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 14:25:13 2022] POST /identity/v3/projects => generated 377 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 14:25:14.055208 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac353df-db84-432b-9816-3fac66674491 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:25:14.066036 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac353df-db84-432b-9816-3fac66674491 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:25:14.129293 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff5dfd2-6bda-4c7e-8106-feb18a980ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:25:14.129958 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff5dfd2-6bda-4c7e-8106-feb18a980ca1 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:14.130243 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff5dfd2-6bda-4c7e-8106-feb18a980ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:14.130521 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff5dfd2-6bda-4c7e-8106-feb18a980ca1 admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:14.131201 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff5dfd2-6bda-4c7e-8106-feb18a980ca1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:25:14.133115 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff5dfd2-6bda-4c7e-8106-feb18a980ca1 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:25:14.155887 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1357/2724] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 14:25:14 2022] POST /identity/v3/users => generated 295 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:25:14.160339 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a232d2fe-cfd3-4f58-a319-b29f2d5083d3 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:25:14.171728 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a232d2fe-cfd3-4f58-a319-b29f2d5083d3 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:25:14.176621 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97a614c-d78f-4155-a316-a89a6771cfde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:25:14.177262 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97a614c-d78f-4155-a316-a89a6771cfde admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:14.177540 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97a614c-d78f-4155-a316-a89a6771cfde admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:14.177816 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97a614c-d78f-4155-a316-a89a6771cfde admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:14.178541 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97a614c-d78f-4155-a316-a89a6771cfde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:25:14.180168 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97a614c-d78f-4155-a316-a89a6771cfde admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:25:14.191490 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1368/2725] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:25:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:25:14.196267 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff5dfd2-6bda-4c7e-8106-feb18a980ca1 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:25:14.206277 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff5dfd2-6bda-4c7e-8106-feb18a980ca1 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:25:14.210864 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b3aed8-e6cc-4145-a36e-b607556acc2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:25:14.211521 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b3aed8-e6cc-4145-a36e-b607556acc2d admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:14.211797 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b3aed8-e6cc-4145-a36e-b607556acc2d admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:14.212068 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b3aed8-e6cc-4145-a36e-b607556acc2d admin admin] PATH_INFO: `/v3/projects/a810cb6514f648f5b7a3e0f6c22a6d05/users/3ab26b8e4c4c4a4e8c9637bc86737684/roles/6e0201a9a6484f7d8b112b1eafff70bd` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:14.227869 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b3aed8-e6cc-4145-a36e-b607556acc2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a810cb6514f648f5b7a3e0f6c22a6d05, user_id=3ab26b8e4c4c4a4e8c9637bc86737684, role_id=6e0201a9a6484f7d8b112b1eafff70bd)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:25:14.231097 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b3aed8-e6cc-4145-a36e-b607556acc2d admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:25:14.245395 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1358/2726] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 14:25:14 2022] PUT /identity/v3/projects/a810cb6514f648f5b7a3e0f6c22a6d05/users/3ab26b8e4c4c4a4e8c9637bc86737684/roles/6e0201a9a6484f7d8b112b1eafff70bd => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:25:14.249513 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97a614c-d78f-4155-a316-a89a6771cfde admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:25:14.260199 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97a614c-d78f-4155-a316-a89a6771cfde admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:25:14.286796 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:25:14.330857 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fae5b54-5ba0-464e-b319-47d463ac0a0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:25:14.331482 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fae5b54-5ba0-464e-b319-47d463ac0a0d admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:14.331765 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fae5b54-5ba0-464e-b319-47d463ac0a0d admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:14.332042 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fae5b54-5ba0-464e-b319-47d463ac0a0d admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:14.332737 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fae5b54-5ba0-464e-b319-47d463ac0a0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:25:14.334348 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fae5b54-5ba0-464e-b319-47d463ac0a0d admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:25:14.345063 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1369/2727] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:25:14 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:25:14.349263 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b3aed8-e6cc-4145-a36e-b607556acc2d admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:25:14.354901 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:25:14.361335 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b3aed8-e6cc-4145-a36e-b607556acc2d admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:25:14.369381 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d61f2d2-4aef-4f55-b1b2-821103555e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:25:14.370038 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d61f2d2-4aef-4f55-b1b2-821103555e09 admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:14.370357 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d61f2d2-4aef-4f55-b1b2-821103555e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:14.370636 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d61f2d2-4aef-4f55-b1b2-821103555e09 admin admin] PATH_INFO: `/v3/projects/a810cb6514f648f5b7a3e0f6c22a6d05/users/3ab26b8e4c4c4a4e8c9637bc86737684/roles/6e0201a9a6484f7d8b112b1eafff70bd` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:14.372639 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d61f2d2-4aef-4f55-b1b2-821103555e09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a810cb6514f648f5b7a3e0f6c22a6d05, user_id=3ab26b8e4c4c4a4e8c9637bc86737684, role_id=6e0201a9a6484f7d8b112b1eafff70bd)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:25:14.375303 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d61f2d2-4aef-4f55-b1b2-821103555e09 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:25:14.387379 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1359/2728] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 14:25:14 2022] PUT /identity/v3/projects/a810cb6514f648f5b7a3e0f6c22a6d05/users/3ab26b8e4c4c4a4e8c9637bc86737684/roles/6e0201a9a6484f7d8b112b1eafff70bd => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:25:14.398971 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883d9d26-b7c0-4632-a66b-bed42b0bab5a None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:14.399260 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883d9d26-b7c0-4632-a66b-bed42b0bab5a None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:14.399541 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883d9d26-b7c0-4632-a66b-bed42b0bab5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:14.427958 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-883d9d26-b7c0-4632-a66b-bed42b0bab5a None None] MFA Rules not processed for user `3ab26b8e4c4c4a4e8c9637bc86737684`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:25:14.429157 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-883d9d26-b7c0-4632-a66b-bed42b0bab5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:25:14.498132 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1370/2729] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:25:14 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:25:14.510455 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d61f2d2-4aef-4f55-b1b2-821103555e09 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:25:14.512870 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-9d61f2d2-4aef-4f55-b1b2-821103555e09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:25:14.574111 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d61f2d2-4aef-4f55-b1b2-821103555e09 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:25:14.577943 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10db40b6-271b-46d2-8ef8-01659cb92c6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:25:14.578583 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10db40b6-271b-46d2-8ef8-01659cb92c6c service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:14.578890 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10db40b6-271b-46d2-8ef8-01659cb92c6c service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:14.579164 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10db40b6-271b-46d2-8ef8-01659cb92c6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:14.637527 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10db40b6-271b-46d2-8ef8-01659cb92c6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:25:14.638778 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10db40b6-271b-46d2-8ef8-01659cb92c6c service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:25:14.647715 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1360/2730] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:25:14 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:25:20.318620 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883d9d26-b7c0-4632-a66b-bed42b0bab5a None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:25:20.328498 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883d9d26-b7c0-4632-a66b-bed42b0bab5a None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:25:20.332272 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc61ae2b-59c9-4d54-ac6b-eb031ce7d5d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:25:20.332888 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc61ae2b-59c9-4d54-ac6b-eb031ce7d5d2 service cinder] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:20.333170 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc61ae2b-59c9-4d54-ac6b-eb031ce7d5d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:20.333503 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc61ae2b-59c9-4d54-ac6b-eb031ce7d5d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:20.342936 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc61ae2b-59c9-4d54-ac6b-eb031ce7d5d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:25:20.344578 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc61ae2b-59c9-4d54-ac6b-eb031ce7d5d2 service cinder] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:25:20.354122 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1371/2731] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:25:20 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:25:24.644187 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=52,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:25:30.349829 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=58 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:25:31.610587 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0843a04b-b9b5-40cc-9c38-c1c4a47f4a19 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:31.611589 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0843a04b-b9b5-40cc-9c38-c1c4a47f4a19 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:31.611589 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0843a04b-b9b5-40cc-9c38-c1c4a47f4a19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:31.630334 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-0843a04b-b9b5-40cc-9c38-c1c4a47f4a19 None None] MFA Rules not processed for user `7745969be55b487dbb15b216a3016731`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:25:31.631661 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-0843a04b-b9b5-40cc-9c38-c1c4a47f4a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:25:31.695841 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1361/2732] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:25:31 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:25:41.687321 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=25 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:25:45.573219 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc61ae2b-59c9-4d54-ac6b-eb031ce7d5d2 service cinder] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:25:45.636782 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc61ae2b-59c9-4d54-ac6b-eb031ce7d5d2 service cinder] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:25:45.641370 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745de016-ee2a-4044-9b1d-3b8c9aac2d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:25:45.642036 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745de016-ee2a-4044-9b1d-3b8c9aac2d06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:45.642320 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745de016-ee2a-4044-9b1d-3b8c9aac2d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:45.642601 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745de016-ee2a-4044-9b1d-3b8c9aac2d06 admin admin] PATH_INFO: `/v3/users/42bc9208462947009c2e35b23e243f59` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:45.643762 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745de016-ee2a-4044-9b1d-3b8c9aac2d06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42bc9208462947009c2e35b23e243f59)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:25:45.645834 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745de016-ee2a-4044-9b1d-3b8c9aac2d06 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:25:45.695021 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-745de016-ee2a-4044-9b1d-3b8c9aac2d06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42bc9208462947009c2e35b23e243f59'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:25:45.702668 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-745de016-ee2a-4044-9b1d-3b8c9aac2d06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42bc9208462947009c2e35b23e243f59'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:25:45.711517 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-745de016-ee2a-4044-9b1d-3b8c9aac2d06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42bc9208462947009c2e35b23e243f59'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:25:45.712310 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-745de016-ee2a-4044-9b1d-3b8c9aac2d06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42bc9208462947009c2e35b23e243f59'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:25:45.713045 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-745de016-ee2a-4044-9b1d-3b8c9aac2d06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42bc9208462947009c2e35b23e243f59'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:25:45.732076 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1372/2733] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 14:25:45 2022] DELETE /identity/v3/users/42bc9208462947009c2e35b23e243f59 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:25:45.736661 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7539825-8e65-4ea3-a0f2-57e200ae0c3b None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:45.737413 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7539825-8e65-4ea3-a0f2-57e200ae0c3b None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:45.737413 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7539825-8e65-4ea3-a0f2-57e200ae0c3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:45.756285 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-a7539825-8e65-4ea3-a0f2-57e200ae0c3b None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:25:45.757490 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-a7539825-8e65-4ea3-a0f2-57e200ae0c3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:25:45.817553 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1362/2734] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:25:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:25:45.827605 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745de016-ee2a-4044-9b1d-3b8c9aac2d06 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:25:45.829111 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-745de016-ee2a-4044-9b1d-3b8c9aac2d06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:25:45.885693 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745de016-ee2a-4044-9b1d-3b8c9aac2d06 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:25:45.889854 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a958ae7-1f3b-4ec2-b46d-a786602a8a3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:25:45.890467 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a958ae7-1f3b-4ec2-b46d-a786602a8a3d service neutron] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:45.890753 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a958ae7-1f3b-4ec2-b46d-a786602a8a3d service neutron] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:45.891025 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a958ae7-1f3b-4ec2-b46d-a786602a8a3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:45.900126 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a958ae7-1f3b-4ec2-b46d-a786602a8a3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:25:45.901614 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a958ae7-1f3b-4ec2-b46d-a786602a8a3d service neutron] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:25:45.912030 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1373/2735] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:25:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:25:46.196162 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7539825-8e65-4ea3-a0f2-57e200ae0c3b None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:25:46.197725 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-a7539825-8e65-4ea3-a0f2-57e200ae0c3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:25:46.215609 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7539825-8e65-4ea3-a0f2-57e200ae0c3b None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:25:46.221411 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2effd0-e093-4361-a23e-1d8e88bbdacb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:25:46.222118 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2effd0-e093-4361-a23e-1d8e88bbdacb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:46.222436 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2effd0-e093-4361-a23e-1d8e88bbdacb admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:46.222768 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2effd0-e093-4361-a23e-1d8e88bbdacb admin admin] PATH_INFO: `/v3/projects/699b3d1746c84b18a6f994a13c2c504f` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:46.223952 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2effd0-e093-4361-a23e-1d8e88bbdacb admin admin] RBAC: Authorizing `identity:delete_project(project_id=699b3d1746c84b18a6f994a13c2c504f)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:25:46.226051 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2effd0-e093-4361-a23e-1d8e88bbdacb admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:25:46.284546 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-cd2effd0-e093-4361-a23e-1d8e88bbdacb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '699b3d1746c84b18a6f994a13c2c504f', 'request_id': 'req-cd2effd0-e093-4361-a23e-1d8e88bbdacb'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:25:46.291734 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-cd2effd0-e093-4361-a23e-1d8e88bbdacb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '699b3d1746c84b18a6f994a13c2c504f', 'request_id': 'req-cd2effd0-e093-4361-a23e-1d8e88bbdacb'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:25:46.301189 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-cd2effd0-e093-4361-a23e-1d8e88bbdacb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '699b3d1746c84b18a6f994a13c2c504f', 'request_id': 'req-cd2effd0-e093-4361-a23e-1d8e88bbdacb'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:25:46.317775 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-cd2effd0-e093-4361-a23e-1d8e88bbdacb admin admin] The token cache is being invalidate because project 699b3d1746c84b18a6f994a13c2c504f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:25:46.318134 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-cd2effd0-e093-4361-a23e-1d8e88bbdacb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:25:46.318698 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-cd2effd0-e093-4361-a23e-1d8e88bbdacb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:25:46.320207 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1363/2736] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 14:25:46 2022] DELETE /identity/v3/projects/699b3d1746c84b18a6f994a13c2c504f => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:25:46.324240 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a958ae7-1f3b-4ec2-b46d-a786602a8a3d service neutron] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:25:46.334361 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a958ae7-1f3b-4ec2-b46d-a786602a8a3d service neutron] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:25:46.352709 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:25:46.395759 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28166b71-3ede-41b9-b294-a5de3823d049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:25:46.396401 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28166b71-3ede-41b9-b294-a5de3823d049 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:46.396679 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28166b71-3ede-41b9-b294-a5de3823d049 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:46.396949 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28166b71-3ede-41b9-b294-a5de3823d049 admin admin] PATH_INFO: `/v3/users/3ab26b8e4c4c4a4e8c9637bc86737684` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:46.398101 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28166b71-3ede-41b9-b294-a5de3823d049 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ab26b8e4c4c4a4e8c9637bc86737684)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:25:46.400006 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28166b71-3ede-41b9-b294-a5de3823d049 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:25:46.446584 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-28166b71-3ede-41b9-b294-a5de3823d049 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ab26b8e4c4c4a4e8c9637bc86737684'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:25:46.456465 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-28166b71-3ede-41b9-b294-a5de3823d049 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ab26b8e4c4c4a4e8c9637bc86737684'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:25:46.465889 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-28166b71-3ede-41b9-b294-a5de3823d049 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ab26b8e4c4c4a4e8c9637bc86737684'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:25:46.466892 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-28166b71-3ede-41b9-b294-a5de3823d049 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ab26b8e4c4c4a4e8c9637bc86737684'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:25:46.467857 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-28166b71-3ede-41b9-b294-a5de3823d049 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ab26b8e4c4c4a4e8c9637bc86737684'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:25:46.489711 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1374/2737] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 14:25:46 2022] DELETE /identity/v3/users/3ab26b8e4c4c4a4e8c9637bc86737684 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:25:46.777204 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2effd0-e093-4361-a23e-1d8e88bbdacb admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:25:46.779276 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-cd2effd0-e093-4361-a23e-1d8e88bbdacb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:25:46.795780 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2effd0-e093-4361-a23e-1d8e88bbdacb admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:25:46.856802 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3803fb60-bea3-418c-94dd-300c5c74916b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:25:46.857366 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3803fb60-bea3-418c-94dd-300c5c74916b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:46.857712 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3803fb60-bea3-418c-94dd-300c5c74916b admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:46.857990 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3803fb60-bea3-418c-94dd-300c5c74916b admin admin] PATH_INFO: `/v3/projects/a810cb6514f648f5b7a3e0f6c22a6d05` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:46.859248 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3803fb60-bea3-418c-94dd-300c5c74916b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a810cb6514f648f5b7a3e0f6c22a6d05)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:25:46.861175 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3803fb60-bea3-418c-94dd-300c5c74916b admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:25:46.892239 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:25:46.911864 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-3803fb60-bea3-418c-94dd-300c5c74916b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a810cb6514f648f5b7a3e0f6c22a6d05', 'request_id': 'req-3803fb60-bea3-418c-94dd-300c5c74916b'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:25:46.917963 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-3803fb60-bea3-418c-94dd-300c5c74916b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a810cb6514f648f5b7a3e0f6c22a6d05', 'request_id': 'req-3803fb60-bea3-418c-94dd-300c5c74916b'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:25:46.923423 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-3803fb60-bea3-418c-94dd-300c5c74916b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a810cb6514f648f5b7a3e0f6c22a6d05', 'request_id': 'req-3803fb60-bea3-418c-94dd-300c5c74916b'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:25:46.936955 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-3803fb60-bea3-418c-94dd-300c5c74916b admin admin] The token cache is being invalidate because project a810cb6514f648f5b7a3e0f6c22a6d05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:25:46.937260 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-3803fb60-bea3-418c-94dd-300c5c74916b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:25:46.937927 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-3803fb60-bea3-418c-94dd-300c5c74916b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:25:46.939412 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1364/2738] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 14:25:46 2022] DELETE /identity/v3/projects/a810cb6514f648f5b7a3e0f6c22a6d05 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:25:46.944268 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aab776-eba9-44f8-a978-71679e9d657f None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:46.944729 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aab776-eba9-44f8-a978-71679e9d657f None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:46.944840 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aab776-eba9-44f8-a978-71679e9d657f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:46.964510 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-37aab776-eba9-44f8-a978-71679e9d657f None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:25:46.965737 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-37aab776-eba9-44f8-a978-71679e9d657f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:25:47.023969 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1375/2739] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:25:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:25:47.031999 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f142cd5-a7e6-4aeb-9f83-345be9337d1f None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:47.032286 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f142cd5-a7e6-4aeb-9f83-345be9337d1f None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:47.032560 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f142cd5-a7e6-4aeb-9f83-345be9337d1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:47.052132 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-5f142cd5-a7e6-4aeb-9f83-345be9337d1f None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:25:47.053361 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-5f142cd5-a7e6-4aeb-9f83-345be9337d1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:25:47.061767 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1365/2740] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:25:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:25:47.066031 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aab776-eba9-44f8-a978-71679e9d657f None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:25:47.076223 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aab776-eba9-44f8-a978-71679e9d657f None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:25:47.081060 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9033c431-5788-49aa-8b3d-cdcc73c7ff8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:25:47.081708 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9033c431-5788-49aa-8b3d-cdcc73c7ff8c admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:47.081986 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9033c431-5788-49aa-8b3d-cdcc73c7ff8c admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:47.082259 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9033c431-5788-49aa-8b3d-cdcc73c7ff8c admin admin] PATH_INFO: `/v3/domains` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:47.082944 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9033c431-5788-49aa-8b3d-cdcc73c7ff8c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:25:47.083246 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9033c431-5788-49aa-8b3d-cdcc73c7ff8c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:25:47.084811 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9033c431-5788-49aa-8b3d-cdcc73c7ff8c admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:25:47.096036 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1376/2741] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Jul 26 14:25:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:25:47.100334 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f142cd5-a7e6-4aeb-9f83-345be9337d1f None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:25:47.109949 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f142cd5-a7e6-4aeb-9f83-345be9337d1f None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:25:47.114427 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99010875-4891-488b-9d2a-048e4522f340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:25:47.115024 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99010875-4891-488b-9d2a-048e4522f340 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:47.115302 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99010875-4891-488b-9d2a-048e4522f340 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:47.115574 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99010875-4891-488b-9d2a-048e4522f340 admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:47.116240 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99010875-4891-488b-9d2a-048e4522f340 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:25:47.118183 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99010875-4891-488b-9d2a-048e4522f340 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:25:47.147190 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1366/2742] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 14:25:47 2022] POST /identity/v3/projects => generated 391 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 14:25:47.151432 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9033c431-5788-49aa-8b3d-cdcc73c7ff8c admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:25:47.161371 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9033c431-5788-49aa-8b3d-cdcc73c7ff8c admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:25:47.186876 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:25:47.217800 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f9c070-5036-49ba-8f16-2da337842219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:25:47.218494 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f9c070-5036-49ba-8f16-2da337842219 admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:47.218771 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f9c070-5036-49ba-8f16-2da337842219 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:47.219046 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f9c070-5036-49ba-8f16-2da337842219 admin admin] PATH_INFO: `/v3/users` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:47.219694 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f9c070-5036-49ba-8f16-2da337842219 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:25:47.221630 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f9c070-5036-49ba-8f16-2da337842219 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:25:47.244873 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1377/2743] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 14:25:47 2022] POST /identity/v3/users => generated 296 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 14:25:47.249689 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99010875-4891-488b-9d2a-048e4522f340 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:25:47.260392 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99010875-4891-488b-9d2a-048e4522f340 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:25:47.265072 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28807eba-5838-4d30-92df-489233652faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:25:47.265719 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28807eba-5838-4d30-92df-489233652faa admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:47.266007 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28807eba-5838-4d30-92df-489233652faa admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:47.266287 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28807eba-5838-4d30-92df-489233652faa admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:47.266979 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28807eba-5838-4d30-92df-489233652faa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:25:47.268565 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28807eba-5838-4d30-92df-489233652faa admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:25:47.278443 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1367/2744] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:25:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:25:47.282649 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f9c070-5036-49ba-8f16-2da337842219 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:25:47.293082 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f9c070-5036-49ba-8f16-2da337842219 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:25:47.297674 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafdfcfa-a8d1-46c7-95bf-ec1bcb4e34d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:25:47.298283 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafdfcfa-a8d1-46c7-95bf-ec1bcb4e34d7 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:47.298569 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafdfcfa-a8d1-46c7-95bf-ec1bcb4e34d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:47.298846 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafdfcfa-a8d1-46c7-95bf-ec1bcb4e34d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:47.299543 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafdfcfa-a8d1-46c7-95bf-ec1bcb4e34d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:25:47.301114 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafdfcfa-a8d1-46c7-95bf-ec1bcb4e34d7 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:25:47.312246 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1378/2745] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:25:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 14:25:47.316688 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28807eba-5838-4d30-92df-489233652faa admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:25:47.326561 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28807eba-5838-4d30-92df-489233652faa admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:25:47.331270 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3f9d43-3446-42ef-a03e-4cc748a958a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:25:47.331998 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3f9d43-3446-42ef-a03e-4cc748a958a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:47.332351 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3f9d43-3446-42ef-a03e-4cc748a958a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:47.332633 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3f9d43-3446-42ef-a03e-4cc748a958a2 admin admin] PATH_INFO: `/v3/projects/a9575179ea014375a42aeae6359933b3/users/e2b4c225079644c1baedd034eb68e0d8/roles/759f389b221049c4b01faf3b984cd8d7` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:47.348040 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3f9d43-3446-42ef-a03e-4cc748a958a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9575179ea014375a42aeae6359933b3, user_id=e2b4c225079644c1baedd034eb68e0d8, role_id=759f389b221049c4b01faf3b984cd8d7)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:25:47.350687 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3f9d43-3446-42ef-a03e-4cc748a958a2 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:25:47.364291 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1368/2746] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 14:25:47 2022] PUT /identity/v3/projects/a9575179ea014375a42aeae6359933b3/users/e2b4c225079644c1baedd034eb68e0d8/roles/759f389b221049c4b01faf3b984cd8d7 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:25:47.376314 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd181c91-f50a-4400-b6d2-5a030147b32a None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:47.376607 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd181c91-f50a-4400-b6d2-5a030147b32a None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:47.376889 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd181c91-f50a-4400-b6d2-5a030147b32a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:47.404908 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-cd181c91-f50a-4400-b6d2-5a030147b32a None None] MFA Rules not processed for user `e2b4c225079644c1baedd034eb68e0d8`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:25:47.406168 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-cd181c91-f50a-4400-b6d2-5a030147b32a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:25:47.471618 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1379/2747] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:25:47 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:25:47.475670 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3f9d43-3446-42ef-a03e-4cc748a958a2 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:25:47.540475 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3f9d43-3446-42ef-a03e-4cc748a958a2 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:25:47.545635 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4d4a0c-56af-4f38-a49f-29c4c630d6ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:25:47.546244 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4d4a0c-56af-4f38-a49f-29c4c630d6ad admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:47.546523 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4d4a0c-56af-4f38-a49f-29c4c630d6ad admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:47.546801 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4d4a0c-56af-4f38-a49f-29c4c630d6ad admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:47.547589 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4d4a0c-56af-4f38-a49f-29c4c630d6ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:25:47.549401 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4d4a0c-56af-4f38-a49f-29c4c630d6ad admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:25:47.578825 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1369/2748] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 14:25:47 2022] POST /identity/v3/projects => generated 389 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:25:47.583508 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd181c91-f50a-4400-b6d2-5a030147b32a None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:25:47.594868 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd181c91-f50a-4400-b6d2-5a030147b32a None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:25:47.654549 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbb3046-070b-45a0-8cf1-f5550de092b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:25:47.655216 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb3046-070b-45a0-8cf1-f5550de092b9 admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:47.655457 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb3046-070b-45a0-8cf1-f5550de092b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:47.655729 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb3046-070b-45a0-8cf1-f5550de092b9 admin admin] PATH_INFO: `/v3/users` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:47.656402 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbb3046-070b-45a0-8cf1-f5550de092b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:25:47.658329 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbb3046-070b-45a0-8cf1-f5550de092b9 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:25:47.691708 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1380/2749] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 14:25:47 2022] POST /identity/v3/users => generated 301 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 14:25:47.696374 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4d4a0c-56af-4f38-a49f-29c4c630d6ad admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:25:47.707416 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4d4a0c-56af-4f38-a49f-29c4c630d6ad admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:25:47.713668 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c10d78-e16b-46f8-95ce-df84dd4deacb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:25:47.715215 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c10d78-e16b-46f8-95ce-df84dd4deacb admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:47.715215 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c10d78-e16b-46f8-95ce-df84dd4deacb admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:47.715431 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c10d78-e16b-46f8-95ce-df84dd4deacb admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:47.716051 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c10d78-e16b-46f8-95ce-df84dd4deacb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:25:47.717806 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c10d78-e16b-46f8-95ce-df84dd4deacb admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:25:47.736181 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1370/2750] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:25:47 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:25:47.740470 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbb3046-070b-45a0-8cf1-f5550de092b9 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:25:47.750664 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbb3046-070b-45a0-8cf1-f5550de092b9 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:25:47.755609 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b42b7ae-ac17-45e6-a307-eae4327014c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:25:47.756297 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b42b7ae-ac17-45e6-a307-eae4327014c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:47.756584 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b42b7ae-ac17-45e6-a307-eae4327014c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:47.756865 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b42b7ae-ac17-45e6-a307-eae4327014c6 admin admin] PATH_INFO: `/v3/projects/1ead823bf9284a0e8c7bddfde131bc1a/users/47a99cf72d284e0a9e43ccd796d2f95d/roles/6e0201a9a6484f7d8b112b1eafff70bd` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:47.772774 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b42b7ae-ac17-45e6-a307-eae4327014c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ead823bf9284a0e8c7bddfde131bc1a, user_id=47a99cf72d284e0a9e43ccd796d2f95d, role_id=6e0201a9a6484f7d8b112b1eafff70bd)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:25:47.775539 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b42b7ae-ac17-45e6-a307-eae4327014c6 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:25:47.789626 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1381/2751] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 14:25:47 2022] PUT /identity/v3/projects/1ead823bf9284a0e8c7bddfde131bc1a/users/47a99cf72d284e0a9e43ccd796d2f95d/roles/6e0201a9a6484f7d8b112b1eafff70bd => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:25:47.794357 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c10d78-e16b-46f8-95ce-df84dd4deacb admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:25:47.807565 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c10d78-e16b-46f8-95ce-df84dd4deacb admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:25:47.827128 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:25:47.867781 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b214124c-6d4c-48d9-9a32-0720048e2560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:25:47.868395 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b214124c-6d4c-48d9-9a32-0720048e2560 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:47.868676 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b214124c-6d4c-48d9-9a32-0720048e2560 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:47.868948 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b214124c-6d4c-48d9-9a32-0720048e2560 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:47.869661 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b214124c-6d4c-48d9-9a32-0720048e2560 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:25:47.871282 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b214124c-6d4c-48d9-9a32-0720048e2560 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:25:47.881134 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1371/2752] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:25:47 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:25:47.886538 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b42b7ae-ac17-45e6-a307-eae4327014c6 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:25:47.891931 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:25:47.898346 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b42b7ae-ac17-45e6-a307-eae4327014c6 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:25:47.904382 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2babae-5d81-4119-9010-c090c5acf906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:25:47.905053 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2babae-5d81-4119-9010-c090c5acf906 admin admin] REQUEST_METHOD: `PUT` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:47.905336 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2babae-5d81-4119-9010-c090c5acf906 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:47.905658 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2babae-5d81-4119-9010-c090c5acf906 admin admin] PATH_INFO: `/v3/projects/1ead823bf9284a0e8c7bddfde131bc1a/users/47a99cf72d284e0a9e43ccd796d2f95d/roles/6e0201a9a6484f7d8b112b1eafff70bd` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:47.907630 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2babae-5d81-4119-9010-c090c5acf906 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ead823bf9284a0e8c7bddfde131bc1a, user_id=47a99cf72d284e0a9e43ccd796d2f95d, role_id=6e0201a9a6484f7d8b112b1eafff70bd)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:25:47.910273 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2babae-5d81-4119-9010-c090c5acf906 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:25:47.920942 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1382/2753] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 14:25:47 2022] PUT /identity/v3/projects/1ead823bf9284a0e8c7bddfde131bc1a/users/47a99cf72d284e0a9e43ccd796d2f95d/roles/6e0201a9a6484f7d8b112b1eafff70bd => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:25:47.935476 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2906a0-a930-4c3a-afd8-200f99f31d63 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:47.935764 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2906a0-a930-4c3a-afd8-200f99f31d63 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:47.936042 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2906a0-a930-4c3a-afd8-200f99f31d63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:47.963790 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-ab2906a0-a930-4c3a-afd8-200f99f31d63 None None] MFA Rules not processed for user `47a99cf72d284e0a9e43ccd796d2f95d`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:25:47.965593 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-ab2906a0-a930-4c3a-afd8-200f99f31d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:25:48.035945 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1372/2754] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:25:47 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:25:48.046741 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2babae-5d81-4119-9010-c090c5acf906 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:25:48.048520 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-fa2babae-5d81-4119-9010-c090c5acf906 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:25:48.105597 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2babae-5d81-4119-9010-c090c5acf906 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:25:48.109217 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb1f670-f2f6-498a-ac21-a07a4c39b847 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:25:48.109864 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb1f670-f2f6-498a-ac21-a07a4c39b847 service cinder] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:48.110144 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb1f670-f2f6-498a-ac21-a07a4c39b847 service cinder] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:48.110416 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb1f670-f2f6-498a-ac21-a07a4c39b847 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:48.119716 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb1f670-f2f6-498a-ac21-a07a4c39b847 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:25:48.121171 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb1f670-f2f6-498a-ac21-a07a4c39b847 service cinder] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:25:48.131053 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1383/2755] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:25:48 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:25:48.192952 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2906a0-a930-4c3a-afd8-200f99f31d63 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:25:48.194363 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-ab2906a0-a930-4c3a-afd8-200f99f31d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:25:48.206901 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2906a0-a930-4c3a-afd8-200f99f31d63 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:25:48.210473 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c726cb25-c193-460e-87ef-c849dcb6148c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:25:48.211082 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c726cb25-c193-460e-87ef-c849dcb6148c service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:48.211364 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c726cb25-c193-460e-87ef-c849dcb6148c service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:48.211639 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c726cb25-c193-460e-87ef-c849dcb6148c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:48.269398 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c726cb25-c193-460e-87ef-c849dcb6148c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:25:48.270864 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c726cb25-c193-460e-87ef-c849dcb6148c service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:25:48.280189 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1373/2756] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:25:48 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:25:51.608578 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb1f670-f2f6-498a-ac21-a07a4c39b847 service cinder] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:25:51.677571 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb1f670-f2f6-498a-ac21-a07a4c39b847 service cinder] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:25:51.682063 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732ca9fa-63cb-4500-b69a-ddc7a515ff0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:25:51.682707 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732ca9fa-63cb-4500-b69a-ddc7a515ff0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:51.682989 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732ca9fa-63cb-4500-b69a-ddc7a515ff0b admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:51.683265 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732ca9fa-63cb-4500-b69a-ddc7a515ff0b admin admin] PATH_INFO: `/v3/users/e2b4c225079644c1baedd034eb68e0d8` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:51.684422 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732ca9fa-63cb-4500-b69a-ddc7a515ff0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2b4c225079644c1baedd034eb68e0d8)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:25:51.686373 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732ca9fa-63cb-4500-b69a-ddc7a515ff0b admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:25:51.735316 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-732ca9fa-63cb-4500-b69a-ddc7a515ff0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2b4c225079644c1baedd034eb68e0d8'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:25:51.742340 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-732ca9fa-63cb-4500-b69a-ddc7a515ff0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2b4c225079644c1baedd034eb68e0d8'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:25:51.749053 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-732ca9fa-63cb-4500-b69a-ddc7a515ff0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2b4c225079644c1baedd034eb68e0d8'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:25:51.749764 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-732ca9fa-63cb-4500-b69a-ddc7a515ff0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2b4c225079644c1baedd034eb68e0d8'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:25:51.750402 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-732ca9fa-63cb-4500-b69a-ddc7a515ff0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2b4c225079644c1baedd034eb68e0d8'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:25:51.768172 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1384/2757] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 14:25:51 2022] DELETE /identity/v3/users/e2b4c225079644c1baedd034eb68e0d8 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:25:51.772815 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dc714f-0809-46c5-81fd-7c9835c056dc None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:51.773330 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dc714f-0809-46c5-81fd-7c9835c056dc None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:51.773420 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dc714f-0809-46c5-81fd-7c9835c056dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:51.802557 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-68dc714f-0809-46c5-81fd-7c9835c056dc None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:25:51.803753 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-68dc714f-0809-46c5-81fd-7c9835c056dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:25:51.834720 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:25:51.863768 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1374/2758] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:25:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:25:51.873366 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732ca9fa-63cb-4500-b69a-ddc7a515ff0b admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:25:51.874869 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-732ca9fa-63cb-4500-b69a-ddc7a515ff0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:25:51.878544 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:25:51.945094 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732ca9fa-63cb-4500-b69a-ddc7a515ff0b admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:25:51.949445 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151e77ac-4e88-489e-a590-aa0055ec5246 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:25:51.950101 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151e77ac-4e88-489e-a590-aa0055ec5246 service neutron] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:51.950385 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151e77ac-4e88-489e-a590-aa0055ec5246 service neutron] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:51.950661 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151e77ac-4e88-489e-a590-aa0055ec5246 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:51.960426 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151e77ac-4e88-489e-a590-aa0055ec5246 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:25:51.961883 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151e77ac-4e88-489e-a590-aa0055ec5246 service neutron] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:25:51.972544 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1385/2759] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:25:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:25:52.249696 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dc714f-0809-46c5-81fd-7c9835c056dc None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:25:52.251162 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-68dc714f-0809-46c5-81fd-7c9835c056dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:25:52.264318 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dc714f-0809-46c5-81fd-7c9835c056dc None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:25:52.269540 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b817bf40-64f8-46b3-89b5-086b20d21be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:25:52.270205 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b817bf40-64f8-46b3-89b5-086b20d21be1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:52.270504 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b817bf40-64f8-46b3-89b5-086b20d21be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:52.271515 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b817bf40-64f8-46b3-89b5-086b20d21be1 admin admin] PATH_INFO: `/v3/projects/a9575179ea014375a42aeae6359933b3` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:52.272893 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b817bf40-64f8-46b3-89b5-086b20d21be1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9575179ea014375a42aeae6359933b3)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:25:52.274834 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b817bf40-64f8-46b3-89b5-086b20d21be1 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:25:52.329349 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-b817bf40-64f8-46b3-89b5-086b20d21be1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9575179ea014375a42aeae6359933b3', 'request_id': 'req-b817bf40-64f8-46b3-89b5-086b20d21be1'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:25:52.335414 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-b817bf40-64f8-46b3-89b5-086b20d21be1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9575179ea014375a42aeae6359933b3', 'request_id': 'req-b817bf40-64f8-46b3-89b5-086b20d21be1'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:25:52.340855 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-b817bf40-64f8-46b3-89b5-086b20d21be1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9575179ea014375a42aeae6359933b3', 'request_id': 'req-b817bf40-64f8-46b3-89b5-086b20d21be1'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:25:52.352027 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-b817bf40-64f8-46b3-89b5-086b20d21be1 admin admin] The token cache is being invalidate because project a9575179ea014375a42aeae6359933b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:25:52.352312 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-b817bf40-64f8-46b3-89b5-086b20d21be1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:25:52.352944 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-b817bf40-64f8-46b3-89b5-086b20d21be1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:25:52.354472 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1375/2760] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 14:25:52 2022] DELETE /identity/v3/projects/a9575179ea014375a42aeae6359933b3 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:25:52.358411 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151e77ac-4e88-489e-a590-aa0055ec5246 service neutron] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:25:52.369099 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151e77ac-4e88-489e-a590-aa0055ec5246 service neutron] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:25:52.428513 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c60b5d-04d5-4d60-ad30-b3bd42d90be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:25:52.428710 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c60b5d-04d5-4d60-ad30-b3bd42d90be0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:52.428969 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c60b5d-04d5-4d60-ad30-b3bd42d90be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:52.429244 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c60b5d-04d5-4d60-ad30-b3bd42d90be0 admin admin] PATH_INFO: `/v3/users/47a99cf72d284e0a9e43ccd796d2f95d` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:52.430482 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c60b5d-04d5-4d60-ad30-b3bd42d90be0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47a99cf72d284e0a9e43ccd796d2f95d)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:25:52.432387 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c60b5d-04d5-4d60-ad30-b3bd42d90be0 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:25:52.485384 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-92c60b5d-04d5-4d60-ad30-b3bd42d90be0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47a99cf72d284e0a9e43ccd796d2f95d'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:25:52.493090 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-92c60b5d-04d5-4d60-ad30-b3bd42d90be0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47a99cf72d284e0a9e43ccd796d2f95d'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:25:52.501301 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-92c60b5d-04d5-4d60-ad30-b3bd42d90be0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47a99cf72d284e0a9e43ccd796d2f95d'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:25:52.502013 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-92c60b5d-04d5-4d60-ad30-b3bd42d90be0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47a99cf72d284e0a9e43ccd796d2f95d'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:25:52.502631 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-92c60b5d-04d5-4d60-ad30-b3bd42d90be0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47a99cf72d284e0a9e43ccd796d2f95d'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:25:52.520020 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1386/2761] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 14:25:52 2022] DELETE /identity/v3/users/47a99cf72d284e0a9e43ccd796d2f95d => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:25:52.803078 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b817bf40-64f8-46b3-89b5-086b20d21be1 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:25:52.804570 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-b817bf40-64f8-46b3-89b5-086b20d21be1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:25:52.818270 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b817bf40-64f8-46b3-89b5-086b20d21be1 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:25:52.876754 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d6a428-35e9-4b94-ac20-485e5dfc8a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:25:52.877403 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d6a428-35e9-4b94-ac20-485e5dfc8a4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:52.877710 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d6a428-35e9-4b94-ac20-485e5dfc8a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:52.877986 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d6a428-35e9-4b94-ac20-485e5dfc8a4e admin admin] PATH_INFO: `/v3/projects/1ead823bf9284a0e8c7bddfde131bc1a` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:52.879142 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d6a428-35e9-4b94-ac20-485e5dfc8a4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ead823bf9284a0e8c7bddfde131bc1a)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:25:52.881020 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d6a428-35e9-4b94-ac20-485e5dfc8a4e admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:25:52.927211 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-63d6a428-35e9-4b94-ac20-485e5dfc8a4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ead823bf9284a0e8c7bddfde131bc1a', 'request_id': 'req-63d6a428-35e9-4b94-ac20-485e5dfc8a4e'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:25:52.933814 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-63d6a428-35e9-4b94-ac20-485e5dfc8a4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ead823bf9284a0e8c7bddfde131bc1a', 'request_id': 'req-63d6a428-35e9-4b94-ac20-485e5dfc8a4e'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:25:52.941612 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:25:52.944017 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-63d6a428-35e9-4b94-ac20-485e5dfc8a4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ead823bf9284a0e8c7bddfde131bc1a', 'request_id': 'req-63d6a428-35e9-4b94-ac20-485e5dfc8a4e'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:25:52.957907 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-63d6a428-35e9-4b94-ac20-485e5dfc8a4e admin admin] The token cache is being invalidate because project 1ead823bf9284a0e8c7bddfde131bc1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:25:52.958200 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-63d6a428-35e9-4b94-ac20-485e5dfc8a4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:25:52.958821 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-63d6a428-35e9-4b94-ac20-485e5dfc8a4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:25:52.960315 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1376/2762] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 14:25:52 2022] DELETE /identity/v3/projects/1ead823bf9284a0e8c7bddfde131bc1a => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:25:52.966145 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f548acd-3802-4d36-bc78-8786cd65d122 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:52.966608 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f548acd-3802-4d36-bc78-8786cd65d122 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:52.967057 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f548acd-3802-4d36-bc78-8786cd65d122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:52.987980 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-2f548acd-3802-4d36-bc78-8786cd65d122 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:25:52.989290 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-2f548acd-3802-4d36-bc78-8786cd65d122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:25:53.002490 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:25:53.052989 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1387/2763] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:25:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:25:53.060816 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639e1aae-abc2-4f8c-a542-a5fa4d0fd1b9 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:53.061153 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639e1aae-abc2-4f8c-a542-a5fa4d0fd1b9 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:53.061430 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639e1aae-abc2-4f8c-a542-a5fa4d0fd1b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:53.081256 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-639e1aae-abc2-4f8c-a542-a5fa4d0fd1b9 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:25:53.082509 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-639e1aae-abc2-4f8c-a542-a5fa4d0fd1b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:25:53.090947 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1377/2764] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:25:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:25:53.095305 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f548acd-3802-4d36-bc78-8786cd65d122 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:25:53.105726 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f548acd-3802-4d36-bc78-8786cd65d122 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:25:53.110272 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e42bd90-5c47-41b7-9d57-607a3cd8a01b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:25:53.110897 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e42bd90-5c47-41b7-9d57-607a3cd8a01b admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:53.111180 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e42bd90-5c47-41b7-9d57-607a3cd8a01b admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:53.111456 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e42bd90-5c47-41b7-9d57-607a3cd8a01b admin admin] PATH_INFO: `/v3/domains` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:53.112157 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e42bd90-5c47-41b7-9d57-607a3cd8a01b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:25:53.112466 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e42bd90-5c47-41b7-9d57-607a3cd8a01b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:25:53.114100 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e42bd90-5c47-41b7-9d57-607a3cd8a01b admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:25:53.126179 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1388/2765] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Jul 26 14:25:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:25:53.131044 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639e1aae-abc2-4f8c-a542-a5fa4d0fd1b9 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:25:53.141249 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639e1aae-abc2-4f8c-a542-a5fa4d0fd1b9 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:25:53.146075 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c436239-3a3b-4aad-9b4a-0cd64254a714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:25:53.146688 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c436239-3a3b-4aad-9b4a-0cd64254a714 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:53.146966 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c436239-3a3b-4aad-9b4a-0cd64254a714 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:53.147235 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c436239-3a3b-4aad-9b4a-0cd64254a714 admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:53.147947 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c436239-3a3b-4aad-9b4a-0cd64254a714 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:25:53.149916 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c436239-3a3b-4aad-9b4a-0cd64254a714 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:25:53.177345 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1378/2766] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 14:25:53 2022] POST /identity/v3/projects => generated 409 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:25:53.181526 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e42bd90-5c47-41b7-9d57-607a3cd8a01b admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:25:53.192053 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e42bd90-5c47-41b7-9d57-607a3cd8a01b admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:25:53.247246 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf94455-6607-4d24-94cb-b52203b3a2e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:25:53.247855 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf94455-6607-4d24-94cb-b52203b3a2e8 admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:53.248138 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf94455-6607-4d24-94cb-b52203b3a2e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:53.248403 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf94455-6607-4d24-94cb-b52203b3a2e8 admin admin] PATH_INFO: `/v3/users` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:53.249052 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf94455-6607-4d24-94cb-b52203b3a2e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:25:53.251205 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf94455-6607-4d24-94cb-b52203b3a2e8 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:25:53.271770 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1389/2767] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 14:25:53 2022] POST /identity/v3/users => generated 305 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:25:53.276020 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c436239-3a3b-4aad-9b4a-0cd64254a714 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:25:53.286753 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c436239-3a3b-4aad-9b4a-0cd64254a714 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:25:53.291309 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb045e6c-d347-4916-9a00-69dbe76415e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:25:53.291908 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb045e6c-d347-4916-9a00-69dbe76415e1 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:53.292187 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb045e6c-d347-4916-9a00-69dbe76415e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:53.292946 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb045e6c-d347-4916-9a00-69dbe76415e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:53.293692 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb045e6c-d347-4916-9a00-69dbe76415e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:25:53.295264 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb045e6c-d347-4916-9a00-69dbe76415e1 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:25:53.305114 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1379/2768] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:25:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:25:53.309681 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf94455-6607-4d24-94cb-b52203b3a2e8 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:25:53.319381 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf94455-6607-4d24-94cb-b52203b3a2e8 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:25:53.323912 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aae1b12-ab02-4216-ae36-902fc757d4ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:25:53.324508 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aae1b12-ab02-4216-ae36-902fc757d4ff admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:53.324788 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aae1b12-ab02-4216-ae36-902fc757d4ff admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:53.325063 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aae1b12-ab02-4216-ae36-902fc757d4ff admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:53.325808 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aae1b12-ab02-4216-ae36-902fc757d4ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:25:53.327499 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aae1b12-ab02-4216-ae36-902fc757d4ff admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:25:53.338000 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1390/2769] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:25:53 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:25:53.342237 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb045e6c-d347-4916-9a00-69dbe76415e1 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:25:53.352455 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb045e6c-d347-4916-9a00-69dbe76415e1 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:25:53.357643 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7ae4d2-6126-4fa6-b729-3de4c09ec10d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:25:53.358303 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ae4d2-6126-4fa6-b729-3de4c09ec10d admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:53.358588 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ae4d2-6126-4fa6-b729-3de4c09ec10d admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:53.358866 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ae4d2-6126-4fa6-b729-3de4c09ec10d admin admin] PATH_INFO: `/v3/projects/a0676308d4cf4257bb0286761569b1a3/users/48caa72aaf5f40d9b3ae98183fbd549d/roles/759f389b221049c4b01faf3b984cd8d7` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:53.374159 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7ae4d2-6126-4fa6-b729-3de4c09ec10d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0676308d4cf4257bb0286761569b1a3, user_id=48caa72aaf5f40d9b3ae98183fbd549d, role_id=759f389b221049c4b01faf3b984cd8d7)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:25:53.376789 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7ae4d2-6126-4fa6-b729-3de4c09ec10d admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:25:53.390450 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1380/2770] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 14:25:53 2022] PUT /identity/v3/projects/a0676308d4cf4257bb0286761569b1a3/users/48caa72aaf5f40d9b3ae98183fbd549d/roles/759f389b221049c4b01faf3b984cd8d7 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:25:53.401743 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b49352c-3677-45fe-9ad7-b20a4e87472a None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:53.402035 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b49352c-3677-45fe-9ad7-b20a4e87472a None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:53.402313 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b49352c-3677-45fe-9ad7-b20a4e87472a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:53.429527 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-8b49352c-3677-45fe-9ad7-b20a4e87472a None None] MFA Rules not processed for user `48caa72aaf5f40d9b3ae98183fbd549d`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:25:53.430739 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-8b49352c-3677-45fe-9ad7-b20a4e87472a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:25:53.492342 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1391/2771] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:25:53 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:25:53.496976 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7ae4d2-6126-4fa6-b729-3de4c09ec10d admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:25:53.560877 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7ae4d2-6126-4fa6-b729-3de4c09ec10d admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:25:53.571388 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f577153c-0654-40a2-859f-d479bfc8b549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:25:53.572027 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f577153c-0654-40a2-859f-d479bfc8b549 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:53.572313 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f577153c-0654-40a2-859f-d479bfc8b549 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:53.572590 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f577153c-0654-40a2-859f-d479bfc8b549 admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:53.573273 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f577153c-0654-40a2-859f-d479bfc8b549 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:25:53.575341 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f577153c-0654-40a2-859f-d479bfc8b549 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:25:53.604187 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1381/2772] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 14:25:53 2022] POST /identity/v3/projects => generated 409 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:25:53.608959 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b49352c-3677-45fe-9ad7-b20a4e87472a None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:25:53.619844 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b49352c-3677-45fe-9ad7-b20a4e87472a None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:25:53.629348 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:25:53.687659 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868ce4ca-b65b-4005-9bba-6227851b85f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:25:53.687659 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868ce4ca-b65b-4005-9bba-6227851b85f0 admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:53.687910 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868ce4ca-b65b-4005-9bba-6227851b85f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:53.688193 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868ce4ca-b65b-4005-9bba-6227851b85f0 admin admin] PATH_INFO: `/v3/users` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:53.688880 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868ce4ca-b65b-4005-9bba-6227851b85f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:25:53.690821 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868ce4ca-b65b-4005-9bba-6227851b85f0 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:25:53.715397 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1392/2773] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 14:25:53 2022] POST /identity/v3/users => generated 311 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:25:53.716487 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f577153c-0654-40a2-859f-d479bfc8b549 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:25:53.726995 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f577153c-0654-40a2-859f-d479bfc8b549 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:25:53.732280 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1790cea6-2470-468e-8ba7-32bfe01b9b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:25:53.732897 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1790cea6-2470-468e-8ba7-32bfe01b9b3d admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:53.733189 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1790cea6-2470-468e-8ba7-32bfe01b9b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:53.733521 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1790cea6-2470-468e-8ba7-32bfe01b9b3d admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:53.734225 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1790cea6-2470-468e-8ba7-32bfe01b9b3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:25:53.736319 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1790cea6-2470-468e-8ba7-32bfe01b9b3d admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:25:53.747986 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1382/2774] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:25:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:25:53.751880 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868ce4ca-b65b-4005-9bba-6227851b85f0 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:25:53.762307 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868ce4ca-b65b-4005-9bba-6227851b85f0 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:25:53.767019 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772a1ea1-43d1-4962-b903-538a5b3a1f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:25:53.767697 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772a1ea1-43d1-4962-b903-538a5b3a1f0b admin admin] REQUEST_METHOD: `PUT` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:53.767983 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772a1ea1-43d1-4962-b903-538a5b3a1f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:53.768261 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772a1ea1-43d1-4962-b903-538a5b3a1f0b admin admin] PATH_INFO: `/v3/projects/901489107da9459c8f6b0fc96a1b9deb/users/37d60238dded4eeba6abacff587f178a/roles/6e0201a9a6484f7d8b112b1eafff70bd` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:53.785220 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772a1ea1-43d1-4962-b903-538a5b3a1f0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=901489107da9459c8f6b0fc96a1b9deb, user_id=37d60238dded4eeba6abacff587f178a, role_id=6e0201a9a6484f7d8b112b1eafff70bd)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:25:53.787910 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772a1ea1-43d1-4962-b903-538a5b3a1f0b admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:25:53.812298 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1393/2775] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 14:25:53 2022] PUT /identity/v3/projects/901489107da9459c8f6b0fc96a1b9deb/users/37d60238dded4eeba6abacff587f178a/roles/6e0201a9a6484f7d8b112b1eafff70bd => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:25:53.816386 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1790cea6-2470-468e-8ba7-32bfe01b9b3d admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:25:53.826957 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1790cea6-2470-468e-8ba7-32bfe01b9b3d admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:25:53.900757 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a89cdb5-98e6-41bd-b277-f4af7f62f521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:25:53.901384 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a89cdb5-98e6-41bd-b277-f4af7f62f521 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:53.902219 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a89cdb5-98e6-41bd-b277-f4af7f62f521 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:53.902493 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a89cdb5-98e6-41bd-b277-f4af7f62f521 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:53.903185 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a89cdb5-98e6-41bd-b277-f4af7f62f521 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:25:53.904774 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a89cdb5-98e6-41bd-b277-f4af7f62f521 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:25:53.906262 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:25:53.916068 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1383/2776] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:25:53 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:25:53.922878 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772a1ea1-43d1-4962-b903-538a5b3a1f0b admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:25:53.933048 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772a1ea1-43d1-4962-b903-538a5b3a1f0b admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:25:53.938187 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1553bfe8-f8d5-41e0-8354-5b25787c936e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:25:53.938864 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1553bfe8-f8d5-41e0-8354-5b25787c936e admin admin] REQUEST_METHOD: `PUT` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:53.939142 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1553bfe8-f8d5-41e0-8354-5b25787c936e admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:53.939440 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1553bfe8-f8d5-41e0-8354-5b25787c936e admin admin] PATH_INFO: `/v3/projects/901489107da9459c8f6b0fc96a1b9deb/users/37d60238dded4eeba6abacff587f178a/roles/6e0201a9a6484f7d8b112b1eafff70bd` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:53.941386 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1553bfe8-f8d5-41e0-8354-5b25787c936e admin admin] RBAC: Authorizing `identity:create_grant(project_id=901489107da9459c8f6b0fc96a1b9deb, user_id=37d60238dded4eeba6abacff587f178a, role_id=6e0201a9a6484f7d8b112b1eafff70bd)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:25:53.944002 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1553bfe8-f8d5-41e0-8354-5b25787c936e admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:25:53.956194 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1394/2777] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 14:25:53 2022] PUT /identity/v3/projects/901489107da9459c8f6b0fc96a1b9deb/users/37d60238dded4eeba6abacff587f178a/roles/6e0201a9a6484f7d8b112b1eafff70bd => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:25:53.966360 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03521fc8-74e4-4723-a1f0-0fa6251c5ac4 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:53.966649 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03521fc8-74e4-4723-a1f0-0fa6251c5ac4 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:53.966925 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03521fc8-74e4-4723-a1f0-0fa6251c5ac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:53.994601 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-03521fc8-74e4-4723-a1f0-0fa6251c5ac4 None None] MFA Rules not processed for user `37d60238dded4eeba6abacff587f178a`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:25:53.995815 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-03521fc8-74e4-4723-a1f0-0fa6251c5ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:25:54.081199 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1384/2778] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:25:53 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:25:54.093560 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1553bfe8-f8d5-41e0-8354-5b25787c936e admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:25:54.095855 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-1553bfe8-f8d5-41e0-8354-5b25787c936e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:25:54.157685 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1553bfe8-f8d5-41e0-8354-5b25787c936e admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:25:54.161428 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fcafb8-2bcb-4091-a6b7-546b4894979d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:25:54.162089 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fcafb8-2bcb-4091-a6b7-546b4894979d service cinder] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:54.162373 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fcafb8-2bcb-4091-a6b7-546b4894979d service cinder] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:54.162646 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fcafb8-2bcb-4091-a6b7-546b4894979d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:54.172661 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fcafb8-2bcb-4091-a6b7-546b4894979d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:25:54.174127 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fcafb8-2bcb-4091-a6b7-546b4894979d service cinder] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:25:54.184698 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1395/2779] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:25:54 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:25:54.354674 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03521fc8-74e4-4723-a1f0-0fa6251c5ac4 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:25:54.365822 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03521fc8-74e4-4723-a1f0-0fa6251c5ac4 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:25:54.369962 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3543dc9d-c263-4c3d-8fe0-9f5dee092095 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:25:54.371112 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3543dc9d-c263-4c3d-8fe0-9f5dee092095 service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:25:54.371531 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3543dc9d-c263-4c3d-8fe0-9f5dee092095 service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:25:54.371811 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3543dc9d-c263-4c3d-8fe0-9f5dee092095 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:25:54.436127 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3543dc9d-c263-4c3d-8fe0-9f5dee092095 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:25:54.438268 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3543dc9d-c263-4c3d-8fe0-9f5dee092095 service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:25:54.450231 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1385/2780] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:25:54 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:26:01.454830 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fcafb8-2bcb-4091-a6b7-546b4894979d service cinder] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:26:01.456241 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-e3fcafb8-2bcb-4091-a6b7-546b4894979d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:26:01.522390 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fcafb8-2bcb-4091-a6b7-546b4894979d service cinder] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:26:01.526199 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5ef851-6322-409c-9e0e-51a4451e1f1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:26:01.526820 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5ef851-6322-409c-9e0e-51a4451e1f1b service neutron] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:26:01.527098 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5ef851-6322-409c-9e0e-51a4451e1f1b service neutron] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:26:01.527397 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5ef851-6322-409c-9e0e-51a4451e1f1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:26:01.529107 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-7a5ef851-6322-409c-9e0e-51a4451e1f1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:26:01.550725 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:26:01.588897 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5ef851-6322-409c-9e0e-51a4451e1f1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:26:01.590413 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5ef851-6322-409c-9e0e-51a4451e1f1b service neutron] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:26:01.599301 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1396/2781] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:26:01 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:26:02.312241 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3543dc9d-c263-4c3d-8fe0-9f5dee092095 service cinder] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:26:02.314152 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-3543dc9d-c263-4c3d-8fe0-9f5dee092095 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:26:02.408089 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:26:02.413278 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3543dc9d-c263-4c3d-8fe0-9f5dee092095 service cinder] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:26:02.418599 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca9280d-1276-4730-9edc-5e5a52acb9fa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '835cbc13756b41b598142df3e35f5492', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:26:02.419223 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca9280d-1276-4730-9edc-5e5a52acb9fa service placement] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:26:02.419506 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca9280d-1276-4730-9edc-5e5a52acb9fa service placement] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:26:02.419782 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca9280d-1276-4730-9edc-5e5a52acb9fa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:26:02.422304 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-6ca9280d-1276-4730-9edc-5e5a52acb9fa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:26:02.434925 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca9280d-1276-4730-9edc-5e5a52acb9fa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:26:02.436787 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca9280d-1276-4730-9edc-5e5a52acb9fa service placement] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:26:02.450093 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1386/2782] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:26:02 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:26:03.012850 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5ef851-6322-409c-9e0e-51a4451e1f1b service neutron] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:26:03.087504 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5ef851-6322-409c-9e0e-51a4451e1f1b service neutron] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:26:03.092260 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029dae3c-81cf-4065-901d-6580d56d3a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:26:03.092903 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029dae3c-81cf-4065-901d-6580d56d3a28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:26:03.093181 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029dae3c-81cf-4065-901d-6580d56d3a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:26:03.093452 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029dae3c-81cf-4065-901d-6580d56d3a28 admin admin] PATH_INFO: `/v3/users/48caa72aaf5f40d9b3ae98183fbd549d` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:26:03.094662 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029dae3c-81cf-4065-901d-6580d56d3a28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=48caa72aaf5f40d9b3ae98183fbd549d)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:26:03.096553 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029dae3c-81cf-4065-901d-6580d56d3a28 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:26:03.146713 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-029dae3c-81cf-4065-901d-6580d56d3a28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48caa72aaf5f40d9b3ae98183fbd549d'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:26:03.154410 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-029dae3c-81cf-4065-901d-6580d56d3a28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48caa72aaf5f40d9b3ae98183fbd549d'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:26:03.162025 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-029dae3c-81cf-4065-901d-6580d56d3a28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48caa72aaf5f40d9b3ae98183fbd549d'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:26:03.162770 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-029dae3c-81cf-4065-901d-6580d56d3a28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48caa72aaf5f40d9b3ae98183fbd549d'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:26:03.163366 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-029dae3c-81cf-4065-901d-6580d56d3a28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48caa72aaf5f40d9b3ae98183fbd549d'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:26:03.182389 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1397/2783] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 14:26:03 2022] DELETE /identity/v3/users/48caa72aaf5f40d9b3ae98183fbd549d => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:26:03.187562 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9725fb9e-6190-446c-a2c3-6060d083f428 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:26:03.187852 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9725fb9e-6190-446c-a2c3-6060d083f428 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:26:03.188131 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9725fb9e-6190-446c-a2c3-6060d083f428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:26:03.207791 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-9725fb9e-6190-446c-a2c3-6060d083f428 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:26:03.209093 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-9725fb9e-6190-446c-a2c3-6060d083f428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:26:03.272619 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1387/2784] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:26:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:26:03.282541 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029dae3c-81cf-4065-901d-6580d56d3a28 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:26:03.283941 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-029dae3c-81cf-4065-901d-6580d56d3a28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:26:03.333814 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:26:03.348515 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029dae3c-81cf-4065-901d-6580d56d3a28 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:26:03.353104 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae8be0a-97b2-4cb6-ae8d-8b118e8f75f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:26:03.353789 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae8be0a-97b2-4cb6-ae8d-8b118e8f75f9 service neutron] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:26:03.354076 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae8be0a-97b2-4cb6-ae8d-8b118e8f75f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:26:03.354356 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae8be0a-97b2-4cb6-ae8d-8b118e8f75f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:26:03.365345 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae8be0a-97b2-4cb6-ae8d-8b118e8f75f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:26:03.366857 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae8be0a-97b2-4cb6-ae8d-8b118e8f75f9 service neutron] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:26:03.378050 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1398/2785] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:26:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:26:03.659293 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9725fb9e-6190-446c-a2c3-6060d083f428 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:26:03.660910 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-9725fb9e-6190-446c-a2c3-6060d083f428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:26:03.679374 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9725fb9e-6190-446c-a2c3-6060d083f428 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:26:03.684026 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc18ffb5-eb22-4a2d-8291-2696e3f3e019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:26:03.684684 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc18ffb5-eb22-4a2d-8291-2696e3f3e019 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:26:03.684968 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc18ffb5-eb22-4a2d-8291-2696e3f3e019 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:26:03.685247 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc18ffb5-eb22-4a2d-8291-2696e3f3e019 admin admin] PATH_INFO: `/v3/projects/a0676308d4cf4257bb0286761569b1a3` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:26:03.686459 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc18ffb5-eb22-4a2d-8291-2696e3f3e019 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0676308d4cf4257bb0286761569b1a3)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:26:03.688382 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc18ffb5-eb22-4a2d-8291-2696e3f3e019 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:26:03.738009 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-fc18ffb5-eb22-4a2d-8291-2696e3f3e019 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0676308d4cf4257bb0286761569b1a3', 'request_id': 'req-fc18ffb5-eb22-4a2d-8291-2696e3f3e019'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:26:03.744158 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-fc18ffb5-eb22-4a2d-8291-2696e3f3e019 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0676308d4cf4257bb0286761569b1a3', 'request_id': 'req-fc18ffb5-eb22-4a2d-8291-2696e3f3e019'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:26:03.749737 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-fc18ffb5-eb22-4a2d-8291-2696e3f3e019 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0676308d4cf4257bb0286761569b1a3', 'request_id': 'req-fc18ffb5-eb22-4a2d-8291-2696e3f3e019'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:26:03.762718 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-fc18ffb5-eb22-4a2d-8291-2696e3f3e019 admin admin] The token cache is being invalidate because project a0676308d4cf4257bb0286761569b1a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:26:03.763009 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-fc18ffb5-eb22-4a2d-8291-2696e3f3e019 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:26:03.763629 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-fc18ffb5-eb22-4a2d-8291-2696e3f3e019 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:26:03.765139 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1388/2786] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 14:26:03 2022] DELETE /identity/v3/projects/a0676308d4cf4257bb0286761569b1a3 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:26:03.769179 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae8be0a-97b2-4cb6-ae8d-8b118e8f75f9 service neutron] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:26:03.783620 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae8be0a-97b2-4cb6-ae8d-8b118e8f75f9 service neutron] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:26:03.847377 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eabe0d-338c-4538-a146-17b343a0023d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:26:03.847377 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eabe0d-338c-4538-a146-17b343a0023d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:26:03.847377 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eabe0d-338c-4538-a146-17b343a0023d admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:26:03.847377 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eabe0d-338c-4538-a146-17b343a0023d admin admin] PATH_INFO: `/v3/users/37d60238dded4eeba6abacff587f178a` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:26:03.848480 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01eabe0d-338c-4538-a146-17b343a0023d admin admin] RBAC: Authorizing `identity:delete_user(user_id=37d60238dded4eeba6abacff587f178a)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:26:03.850559 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01eabe0d-338c-4538-a146-17b343a0023d admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:26:03.904871 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-01eabe0d-338c-4538-a146-17b343a0023d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37d60238dded4eeba6abacff587f178a'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:26:03.913518 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-01eabe0d-338c-4538-a146-17b343a0023d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37d60238dded4eeba6abacff587f178a'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:26:03.921023 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-01eabe0d-338c-4538-a146-17b343a0023d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37d60238dded4eeba6abacff587f178a'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:26:03.921849 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-01eabe0d-338c-4538-a146-17b343a0023d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37d60238dded4eeba6abacff587f178a'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:26:03.922483 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-01eabe0d-338c-4538-a146-17b343a0023d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37d60238dded4eeba6abacff587f178a'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:26:03.943552 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1399/2787] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 14:26:03 2022] DELETE /identity/v3/users/37d60238dded4eeba6abacff587f178a => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:26:04.263906 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc18ffb5-eb22-4a2d-8291-2696e3f3e019 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:26:04.265387 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-fc18ffb5-eb22-4a2d-8291-2696e3f3e019 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:26:04.290890 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc18ffb5-eb22-4a2d-8291-2696e3f3e019 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:26:04.365797 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75053be8-e2ae-4a7b-9c3d-af68188bcfb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:26:04.366488 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75053be8-e2ae-4a7b-9c3d-af68188bcfb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:26:04.366752 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75053be8-e2ae-4a7b-9c3d-af68188bcfb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:26:04.367082 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75053be8-e2ae-4a7b-9c3d-af68188bcfb0 admin admin] PATH_INFO: `/v3/projects/901489107da9459c8f6b0fc96a1b9deb` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:26:04.368318 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75053be8-e2ae-4a7b-9c3d-af68188bcfb0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=901489107da9459c8f6b0fc96a1b9deb)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:26:04.370733 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75053be8-e2ae-4a7b-9c3d-af68188bcfb0 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:26:04.388240 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:26:04.422464 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-75053be8-e2ae-4a7b-9c3d-af68188bcfb0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '901489107da9459c8f6b0fc96a1b9deb', 'request_id': 'req-75053be8-e2ae-4a7b-9c3d-af68188bcfb0'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:26:04.428761 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-75053be8-e2ae-4a7b-9c3d-af68188bcfb0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '901489107da9459c8f6b0fc96a1b9deb', 'request_id': 'req-75053be8-e2ae-4a7b-9c3d-af68188bcfb0'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:26:04.434988 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-75053be8-e2ae-4a7b-9c3d-af68188bcfb0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '901489107da9459c8f6b0fc96a1b9deb', 'request_id': 'req-75053be8-e2ae-4a7b-9c3d-af68188bcfb0'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:26:04.447018 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-75053be8-e2ae-4a7b-9c3d-af68188bcfb0 admin admin] The token cache is being invalidate because project 901489107da9459c8f6b0fc96a1b9deb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:26:04.447302 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-75053be8-e2ae-4a7b-9c3d-af68188bcfb0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:26:04.447909 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-75053be8-e2ae-4a7b-9c3d-af68188bcfb0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:26:04.452161 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1389/2788] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 14:26:04 2022] DELETE /identity/v3/projects/901489107da9459c8f6b0fc96a1b9deb => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:26:04.457076 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c2a6f5-6422-4f1f-bc05-dfbe7215724a None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:26:04.457738 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c2a6f5-6422-4f1f-bc05-dfbe7215724a None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:26:04.457832 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c2a6f5-6422-4f1f-bc05-dfbe7215724a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:26:04.477338 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-f9c2a6f5-6422-4f1f-bc05-dfbe7215724a None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:26:04.478589 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-f9c2a6f5-6422-4f1f-bc05-dfbe7215724a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:26:04.538668 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1400/2789] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:26:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:26:04.548346 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce55228-dc9f-4403-b52b-cf507db2e0b6 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:26:04.548652 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce55228-dc9f-4403-b52b-cf507db2e0b6 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:26:04.549102 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce55228-dc9f-4403-b52b-cf507db2e0b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:26:04.570549 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-9ce55228-dc9f-4403-b52b-cf507db2e0b6 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:26:04.571756 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-9ce55228-dc9f-4403-b52b-cf507db2e0b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:26:04.579938 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1390/2790] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:26:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:26:04.584191 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c2a6f5-6422-4f1f-bc05-dfbe7215724a None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:26:04.590231 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:26:04.594370 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c2a6f5-6422-4f1f-bc05-dfbe7215724a None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:26:04.600359 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a212ae21-b941-4fc7-9d43-58cce4ff5191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:26:04.601147 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a212ae21-b941-4fc7-9d43-58cce4ff5191 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:26:04.601623 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a212ae21-b941-4fc7-9d43-58cce4ff5191 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:26:04.601899 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a212ae21-b941-4fc7-9d43-58cce4ff5191 admin admin] PATH_INFO: `/v3/domains` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:26:04.602608 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a212ae21-b941-4fc7-9d43-58cce4ff5191 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:26:04.602978 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a212ae21-b941-4fc7-9d43-58cce4ff5191 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:26:04.605202 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a212ae21-b941-4fc7-9d43-58cce4ff5191 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:26:04.617498 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1401/2791] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Jul 26 14:26:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:26:04.622469 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce55228-dc9f-4403-b52b-cf507db2e0b6 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:26:04.631864 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce55228-dc9f-4403-b52b-cf507db2e0b6 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:26:04.636835 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be42bc9-3164-4c8f-b746-d33efddc0258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:26:04.637445 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be42bc9-3164-4c8f-b746-d33efddc0258 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:26:04.637766 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be42bc9-3164-4c8f-b746-d33efddc0258 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:26:04.638043 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be42bc9-3164-4c8f-b746-d33efddc0258 admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:26:04.638735 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be42bc9-3164-4c8f-b746-d33efddc0258 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:26:04.640622 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be42bc9-3164-4c8f-b746-d33efddc0258 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:26:04.668841 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1391/2792] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 14:26:04 2022] POST /identity/v3/projects => generated 365 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 14:26:04.673035 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a212ae21-b941-4fc7-9d43-58cce4ff5191 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:26:04.684002 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a212ae21-b941-4fc7-9d43-58cce4ff5191 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:26:04.743498 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1142a23e-9844-4251-af71-c12cc27782c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:26:04.744126 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1142a23e-9844-4251-af71-c12cc27782c7 admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:26:04.744414 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1142a23e-9844-4251-af71-c12cc27782c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:26:04.744694 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1142a23e-9844-4251-af71-c12cc27782c7 admin admin] PATH_INFO: `/v3/users` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:26:04.746057 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1142a23e-9844-4251-af71-c12cc27782c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:26:04.748006 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1142a23e-9844-4251-af71-c12cc27782c7 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:26:04.772172 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1402/2793] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 14:26:04 2022] POST /identity/v3/users => generated 283 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:26:04.778659 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be42bc9-3164-4c8f-b746-d33efddc0258 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:26:04.791226 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be42bc9-3164-4c8f-b746-d33efddc0258 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:26:04.797629 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4129ecd-9d3e-4d03-8732-824c803f1263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:26:04.798264 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4129ecd-9d3e-4d03-8732-824c803f1263 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:26:04.798600 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4129ecd-9d3e-4d03-8732-824c803f1263 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:26:04.798818 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4129ecd-9d3e-4d03-8732-824c803f1263 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:26:04.799518 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4129ecd-9d3e-4d03-8732-824c803f1263 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:26:04.801111 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4129ecd-9d3e-4d03-8732-824c803f1263 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:26:04.812394 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1392/2794] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:26:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:26:04.816451 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1142a23e-9844-4251-af71-c12cc27782c7 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:26:04.826989 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1142a23e-9844-4251-af71-c12cc27782c7 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:26:04.832281 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2a3c2e-e6eb-4a9e-85ba-9ba7ed2594a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:26:04.832908 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2a3c2e-e6eb-4a9e-85ba-9ba7ed2594a8 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:26:04.833305 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2a3c2e-e6eb-4a9e-85ba-9ba7ed2594a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:26:04.833587 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2a3c2e-e6eb-4a9e-85ba-9ba7ed2594a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:26:04.834190 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2a3c2e-e6eb-4a9e-85ba-9ba7ed2594a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:26:04.835779 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2a3c2e-e6eb-4a9e-85ba-9ba7ed2594a8 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:26:04.847343 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1403/2795] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:26:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:26:04.851922 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4129ecd-9d3e-4d03-8732-824c803f1263 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:26:04.863973 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4129ecd-9d3e-4d03-8732-824c803f1263 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:26:04.868530 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532f6248-539c-4aef-9502-533da00519ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:26:04.869243 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532f6248-539c-4aef-9502-533da00519ed admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:26:04.869452 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532f6248-539c-4aef-9502-533da00519ed admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:26:04.869781 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532f6248-539c-4aef-9502-533da00519ed admin admin] PATH_INFO: `/v3/projects/b78358f25fbf41569b73f7500baee3cc/users/338a1a2e73d84021a91b7a90e8b83b7d/roles/759f389b221049c4b01faf3b984cd8d7` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:26:04.885122 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532f6248-539c-4aef-9502-533da00519ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=b78358f25fbf41569b73f7500baee3cc, user_id=338a1a2e73d84021a91b7a90e8b83b7d, role_id=759f389b221049c4b01faf3b984cd8d7)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:26:04.887800 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532f6248-539c-4aef-9502-533da00519ed admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:26:04.902364 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1393/2796] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 14:26:04 2022] PUT /identity/v3/projects/b78358f25fbf41569b73f7500baee3cc/users/338a1a2e73d84021a91b7a90e8b83b7d/roles/759f389b221049c4b01faf3b984cd8d7 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:26:04.907591 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b59b76-11ea-4f84-9bd2-842f31c96961 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:26:04.907872 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b59b76-11ea-4f84-9bd2-842f31c96961 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:26:04.908159 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b59b76-11ea-4f84-9bd2-842f31c96961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:26:04.927877 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-37b59b76-11ea-4f84-9bd2-842f31c96961 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:26:04.929079 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-37b59b76-11ea-4f84-9bd2-842f31c96961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:26:04.988008 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1404/2797] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:26:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:26:04.997833 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532f6248-539c-4aef-9502-533da00519ed admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:26:04.999500 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-532f6248-539c-4aef-9502-533da00519ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:26:05.067113 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532f6248-539c-4aef-9502-533da00519ed admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:26:05.070980 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ef49c0-8bf1-47ad-bdd5-f96530729e91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:26:05.071680 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ef49c0-8bf1-47ad-bdd5-f96530729e91 service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:26:05.072030 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ef49c0-8bf1-47ad-bdd5-f96530729e91 service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:26:05.072367 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ef49c0-8bf1-47ad-bdd5-f96530729e91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:26:05.082929 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ef49c0-8bf1-47ad-bdd5-f96530729e91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:26:05.085123 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ef49c0-8bf1-47ad-bdd5-f96530729e91 service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:26:05.098167 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1394/2798] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:26:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 14:26:06.094072 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92e562d-9a6e-4d9b-af48-688f1508f312 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:26:06.094807 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92e562d-9a6e-4d9b-af48-688f1508f312 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:26:06.094807 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92e562d-9a6e-4d9b-af48-688f1508f312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:26:06.126627 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-a92e562d-9a6e-4d9b-af48-688f1508f312 None None] MFA Rules not processed for user `338a1a2e73d84021a91b7a90e8b83b7d`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:26:06.127911 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-a92e562d-9a6e-4d9b-af48-688f1508f312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:26:06.235588 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:26:06.242742 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1405/2799] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:26:06 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:26:06.255207 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ef49c0-8bf1-47ad-bdd5-f96530729e91 service neutron] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:26:06.256909 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-c4ef49c0-8bf1-47ad-bdd5-f96530729e91 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:26:06.330307 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:26:06.348480 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ef49c0-8bf1-47ad-bdd5-f96530729e91 service neutron] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:26:06.352503 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9073c392-6e1b-4d51-a550-a74fa05940b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:26:06.353251 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9073c392-6e1b-4d51-a550-a74fa05940b3 service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:26:06.353642 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9073c392-6e1b-4d51-a550-a74fa05940b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:26:06.353986 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9073c392-6e1b-4d51-a550-a74fa05940b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:26:06.364695 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9073c392-6e1b-4d51-a550-a74fa05940b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:26:06.366281 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9073c392-6e1b-4d51-a550-a74fa05940b3 service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:26:06.376691 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1395/2800] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:26:06 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:26:07.773802 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92e562d-9a6e-4d9b-af48-688f1508f312 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:26:07.775430 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-a92e562d-9a6e-4d9b-af48-688f1508f312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:26:07.788458 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92e562d-9a6e-4d9b-af48-688f1508f312 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:26:07.792667 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20dcc0a-d428-47a1-9e1a-01be3f4e8702 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:26:07.793291 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20dcc0a-d428-47a1-9e1a-01be3f4e8702 service neutron] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:26:07.793599 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20dcc0a-d428-47a1-9e1a-01be3f4e8702 service neutron] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:26:07.793874 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20dcc0a-d428-47a1-9e1a-01be3f4e8702 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:26:07.795765 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-d20dcc0a-d428-47a1-9e1a-01be3f4e8702 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:26:07.809351 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20dcc0a-d428-47a1-9e1a-01be3f4e8702 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:26:07.810828 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20dcc0a-d428-47a1-9e1a-01be3f4e8702 service neutron] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:26:07.812913 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54048b78-37a1-4e94-9502-2ce6d2bdaf6f None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:26:07.813540 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54048b78-37a1-4e94-9502-2ce6d2bdaf6f None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:26:07.813629 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54048b78-37a1-4e94-9502-2ce6d2bdaf6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:26:07.822017 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1406/2801] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:26:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:26:07.834991 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-54048b78-37a1-4e94-9502-2ce6d2bdaf6f None None] MFA Rules not processed for user `338a1a2e73d84021a91b7a90e8b83b7d`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:26:07.836217 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-54048b78-37a1-4e94-9502-2ce6d2bdaf6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:26:07.844117 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1396/2802] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:26:07 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:26:11.324541 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20dcc0a-d428-47a1-9e1a-01be3f4e8702 service neutron] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:26:11.326018 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-d20dcc0a-d428-47a1-9e1a-01be3f4e8702 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:26:11.442854 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20dcc0a-d428-47a1-9e1a-01be3f4e8702 service neutron] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:26:11.448605 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0273110-6a67-4335-8131-0f62624fc79a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '835cbc13756b41b598142df3e35f5492', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:26:11.449341 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0273110-6a67-4335-8131-0f62624fc79a service placement] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:26:11.449741 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0273110-6a67-4335-8131-0f62624fc79a service placement] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:26:11.450090 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0273110-6a67-4335-8131-0f62624fc79a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:26:11.452513 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-e0273110-6a67-4335-8131-0f62624fc79a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:26:11.465748 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0273110-6a67-4335-8131-0f62624fc79a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:26:11.467676 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0273110-6a67-4335-8131-0f62624fc79a service placement] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:26:11.480066 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1407/2803] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:26:11 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:26:15.124896 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54048b78-37a1-4e94-9502-2ce6d2bdaf6f None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:26:15.126404 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-54048b78-37a1-4e94-9502-2ce6d2bdaf6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:26:15.141485 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54048b78-37a1-4e94-9502-2ce6d2bdaf6f None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:26:15.145996 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c0898c-ebae-4af0-b98b-64368520edf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:26:15.146639 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c0898c-ebae-4af0-b98b-64368520edf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:26:15.146920 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c0898c-ebae-4af0-b98b-64368520edf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:26:15.147196 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c0898c-ebae-4af0-b98b-64368520edf0 admin admin] PATH_INFO: `/v3/users/d18cab03b71a45dc812dce21798ae806` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:26:15.148373 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c0898c-ebae-4af0-b98b-64368520edf0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d18cab03b71a45dc812dce21798ae806)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:26:15.150315 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c0898c-ebae-4af0-b98b-64368520edf0 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:26:15.203615 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-b3c0898c-ebae-4af0-b98b-64368520edf0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd18cab03b71a45dc812dce21798ae806'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:26:15.204408 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-b3c0898c-ebae-4af0-b98b-64368520edf0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd18cab03b71a45dc812dce21798ae806'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:26:15.211444 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-b3c0898c-ebae-4af0-b98b-64368520edf0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd18cab03b71a45dc812dce21798ae806'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:26:15.212079 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-b3c0898c-ebae-4af0-b98b-64368520edf0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd18cab03b71a45dc812dce21798ae806'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:26:15.223858 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-b3c0898c-ebae-4af0-b98b-64368520edf0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd18cab03b71a45dc812dce21798ae806'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:26:15.237804 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1397/2804] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 14:26:15 2022] DELETE /identity/v3/users/d18cab03b71a45dc812dce21798ae806 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:26:15.438957 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0273110-6a67-4335-8131-0f62624fc79a service placement] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:26:15.463041 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0273110-6a67-4335-8131-0f62624fc79a service placement] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:26:15.523853 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44372a19-3256-4110-b878-83a06ed64e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:26:15.524511 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44372a19-3256-4110-b878-83a06ed64e72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:26:15.524791 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44372a19-3256-4110-b878-83a06ed64e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:26:15.525095 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44372a19-3256-4110-b878-83a06ed64e72 admin admin] PATH_INFO: `/v3/projects/c85546b1cfa54a55896b1f28cd3b983a` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:26:15.526881 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44372a19-3256-4110-b878-83a06ed64e72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c85546b1cfa54a55896b1f28cd3b983a)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:26:15.528693 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44372a19-3256-4110-b878-83a06ed64e72 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:26:15.579313 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-44372a19-3256-4110-b878-83a06ed64e72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c85546b1cfa54a55896b1f28cd3b983a', 'request_id': 'req-44372a19-3256-4110-b878-83a06ed64e72'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:26:15.586646 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-44372a19-3256-4110-b878-83a06ed64e72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c85546b1cfa54a55896b1f28cd3b983a', 'request_id': 'req-44372a19-3256-4110-b878-83a06ed64e72'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:26:15.593609 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-44372a19-3256-4110-b878-83a06ed64e72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c85546b1cfa54a55896b1f28cd3b983a', 'request_id': 'req-44372a19-3256-4110-b878-83a06ed64e72'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:26:15.605787 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-44372a19-3256-4110-b878-83a06ed64e72 admin admin] The token cache is being invalidate because project c85546b1cfa54a55896b1f28cd3b983a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86933) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:26:15.606074 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-44372a19-3256-4110-b878-83a06ed64e72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:26:15.606722 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-44372a19-3256-4110-b878-83a06ed64e72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:26:15.608240 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1408/2805] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 14:26:15 2022] DELETE /identity/v3/projects/c85546b1cfa54a55896b1f28cd3b983a => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:26:15.612287 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c0898c-ebae-4af0-b98b-64368520edf0 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:26:15.613740 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-b3c0898c-ebae-4af0-b98b-64368520edf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:26:15.627340 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c0898c-ebae-4af0-b98b-64368520edf0 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:26:15.685430 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ab028f-7a93-432d-9d76-86b853a20ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:26:15.686283 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab028f-7a93-432d-9d76-86b853a20ef0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:26:15.686606 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab028f-7a93-432d-9d76-86b853a20ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:26:15.686923 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab028f-7a93-432d-9d76-86b853a20ef0 admin admin] PATH_INFO: `/v3/users/83d3ef25325c4fc6a0b302d459c4c617` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:26:15.688303 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ab028f-7a93-432d-9d76-86b853a20ef0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83d3ef25325c4fc6a0b302d459c4c617)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:26:15.690734 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ab028f-7a93-432d-9d76-86b853a20ef0 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:26:15.738493 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-02ab028f-7a93-432d-9d76-86b853a20ef0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83d3ef25325c4fc6a0b302d459c4c617'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:26:15.739256 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-02ab028f-7a93-432d-9d76-86b853a20ef0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83d3ef25325c4fc6a0b302d459c4c617'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:26:15.743873 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:26:15.747666 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-02ab028f-7a93-432d-9d76-86b853a20ef0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83d3ef25325c4fc6a0b302d459c4c617'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:26:15.748702 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-02ab028f-7a93-432d-9d76-86b853a20ef0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83d3ef25325c4fc6a0b302d459c4c617'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:26:15.763576 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-02ab028f-7a93-432d-9d76-86b853a20ef0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83d3ef25325c4fc6a0b302d459c4c617'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:26:15.778804 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1398/2806] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 14:26:15 2022] DELETE /identity/v3/users/83d3ef25325c4fc6a0b302d459c4c617 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:26:15.958324 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44372a19-3256-4110-b878-83a06ed64e72 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:26:15.960105 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-44372a19-3256-4110-b878-83a06ed64e72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:26:15.999661 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44372a19-3256-4110-b878-83a06ed64e72 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:26:16.004159 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:26:16.083600 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8a15c8-0c50-474d-8ba1-7c3482ff93c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:26:16.084249 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8a15c8-0c50-474d-8ba1-7c3482ff93c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:26:16.084530 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8a15c8-0c50-474d-8ba1-7c3482ff93c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:26:16.084809 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8a15c8-0c50-474d-8ba1-7c3482ff93c3 admin admin] PATH_INFO: `/v3/projects/426041dec62c40db96d4a4c4f0cde3c7` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:26:16.086011 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8a15c8-0c50-474d-8ba1-7c3482ff93c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=426041dec62c40db96d4a4c4f0cde3c7)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:26:16.087909 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8a15c8-0c50-474d-8ba1-7c3482ff93c3 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:26:16.135502 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-da8a15c8-0c50-474d-8ba1-7c3482ff93c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '426041dec62c40db96d4a4c4f0cde3c7', 'request_id': 'req-da8a15c8-0c50-474d-8ba1-7c3482ff93c3'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:26:16.144562 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-da8a15c8-0c50-474d-8ba1-7c3482ff93c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '426041dec62c40db96d4a4c4f0cde3c7', 'request_id': 'req-da8a15c8-0c50-474d-8ba1-7c3482ff93c3'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:26:16.151809 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-da8a15c8-0c50-474d-8ba1-7c3482ff93c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '426041dec62c40db96d4a4c4f0cde3c7', 'request_id': 'req-da8a15c8-0c50-474d-8ba1-7c3482ff93c3'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:26:16.162866 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-da8a15c8-0c50-474d-8ba1-7c3482ff93c3 admin admin] The token cache is being invalidate because project 426041dec62c40db96d4a4c4f0cde3c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86933) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:26:16.163153 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-da8a15c8-0c50-474d-8ba1-7c3482ff93c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:26:16.163871 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-da8a15c8-0c50-474d-8ba1-7c3482ff93c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:26:16.165377 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1409/2807] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 14:26:15 2022] DELETE /identity/v3/projects/426041dec62c40db96d4a4c4f0cde3c7 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:26:23.802967 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fdb4c8-acd1-439b-8635-437a4ffa3bf6 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:26:23.803506 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fdb4c8-acd1-439b-8635-437a4ffa3bf6 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:26:23.803506 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fdb4c8-acd1-439b-8635-437a4ffa3bf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:26:23.823009 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-d4fdb4c8-acd1-439b-8635-437a4ffa3bf6 None None] MFA Rules not processed for user `338a1a2e73d84021a91b7a90e8b83b7d`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:26:23.824864 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-d4fdb4c8-acd1-439b-8635-437a4ffa3bf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:26:23.881693 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1399/2808] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:26:23 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:26:26.159801 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=48,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:26:26.268524 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8a15c8-0c50-474d-8ba1-7c3482ff93c3 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:26:26.270034 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-da8a15c8-0c50-474d-8ba1-7c3482ff93c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:26:26.328130 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8a15c8-0c50-474d-8ba1-7c3482ff93c3 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:26:26.331672 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab5e0ae-5535-4c7d-af26-e193411b8fcc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:26:26.332285 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab5e0ae-5535-4c7d-af26-e193411b8fcc service neutron] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:26:26.332565 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab5e0ae-5535-4c7d-af26-e193411b8fcc service neutron] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:26:26.332837 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab5e0ae-5535-4c7d-af26-e193411b8fcc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:26:26.334589 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-6ab5e0ae-5535-4c7d-af26-e193411b8fcc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:26:26.344255 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab5e0ae-5535-4c7d-af26-e193411b8fcc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:26:26.345821 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab5e0ae-5535-4c7d-af26-e193411b8fcc service neutron] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:26:26.354478 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1410/2809] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:26:26 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:26:33.874322 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=35,keystone:DELETE=3,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:26:36.350469 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=26 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:26:37.786645 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fdb4c8-acd1-439b-8635-437a4ffa3bf6 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:26:37.796292 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fdb4c8-acd1-439b-8635-437a4ffa3bf6 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:26:37.799866 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d74a10-b2a1-41f6-a4cb-27831888f676 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:26:37.800477 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d74a10-b2a1-41f6-a4cb-27831888f676 service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:26:37.800758 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d74a10-b2a1-41f6-a4cb-27831888f676 service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:26:37.801033 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d74a10-b2a1-41f6-a4cb-27831888f676 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:26:37.802779 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-c8d74a10-b2a1-41f6-a4cb-27831888f676 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:26:37.864387 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d74a10-b2a1-41f6-a4cb-27831888f676 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:26:37.865842 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d74a10-b2a1-41f6-a4cb-27831888f676 service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:26:37.875969 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1400/2810] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:26:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:26:40.904535 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab5e0ae-5535-4c7d-af26-e193411b8fcc service neutron] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:26:40.907738 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-6ab5e0ae-5535-4c7d-af26-e193411b8fcc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:26:40.976523 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d74a10-b2a1-41f6-a4cb-27831888f676 service neutron] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:26:40.978575 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-c8d74a10-b2a1-41f6-a4cb-27831888f676 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:26:41.018420 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d74a10-b2a1-41f6-a4cb-27831888f676 service neutron] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:26:41.022128 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab5e0ae-5535-4c7d-af26-e193411b8fcc service neutron] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:26:41.023023 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36acc731-1000-4477-a9ba-8e6b32562277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:26:41.023699 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36acc731-1000-4477-a9ba-8e6b32562277 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:26:41.023980 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36acc731-1000-4477-a9ba-8e6b32562277 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:26:41.024256 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36acc731-1000-4477-a9ba-8e6b32562277 admin admin] PATH_INFO: `/v3/users/56db6603f359477885b5151dec71c096` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:26:41.028656 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5359f4c5-968c-4f0f-90a8-764106f1aa5d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '835cbc13756b41b598142df3e35f5492', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:26:41.029280 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5359f4c5-968c-4f0f-90a8-764106f1aa5d service placement] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:26:41.029607 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5359f4c5-968c-4f0f-90a8-764106f1aa5d service placement] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:26:41.029888 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5359f4c5-968c-4f0f-90a8-764106f1aa5d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:26:41.032323 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-5359f4c5-968c-4f0f-90a8-764106f1aa5d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:26:41.040256 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36acc731-1000-4477-a9ba-8e6b32562277 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56db6603f359477885b5151dec71c096)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:26:41.043104 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36acc731-1000-4477-a9ba-8e6b32562277 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:26:41.049410 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5359f4c5-968c-4f0f-90a8-764106f1aa5d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:26:41.051063 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5359f4c5-968c-4f0f-90a8-764106f1aa5d service placement] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:26:41.061396 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1411/2811] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:26:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:26:41.098553 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-36acc731-1000-4477-a9ba-8e6b32562277 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56db6603f359477885b5151dec71c096'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:26:41.098553 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-36acc731-1000-4477-a9ba-8e6b32562277 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56db6603f359477885b5151dec71c096'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:26:41.102300 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-36acc731-1000-4477-a9ba-8e6b32562277 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56db6603f359477885b5151dec71c096'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:26:41.102942 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-36acc731-1000-4477-a9ba-8e6b32562277 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56db6603f359477885b5151dec71c096'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:26:41.114617 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-36acc731-1000-4477-a9ba-8e6b32562277 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56db6603f359477885b5151dec71c096'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:26:41.127334 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1401/2812] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 14:26:40 2022] DELETE /identity/v3/users/56db6603f359477885b5151dec71c096 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:26:41.302488 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5359f4c5-968c-4f0f-90a8-764106f1aa5d service placement] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:26:41.314988 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5359f4c5-968c-4f0f-90a8-764106f1aa5d service placement] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:26:41.373003 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599abaa5-a14c-4a43-bcd6-df9b30b59879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:26:41.373681 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599abaa5-a14c-4a43-bcd6-df9b30b59879 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:26:41.373966 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599abaa5-a14c-4a43-bcd6-df9b30b59879 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:26:41.374242 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599abaa5-a14c-4a43-bcd6-df9b30b59879 admin admin] PATH_INFO: `/v3/projects/f4814f42888d4554a082b2bdb2036ac8` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:26:41.375409 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599abaa5-a14c-4a43-bcd6-df9b30b59879 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4814f42888d4554a082b2bdb2036ac8)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:26:41.377276 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599abaa5-a14c-4a43-bcd6-df9b30b59879 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:26:41.424388 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-599abaa5-a14c-4a43-bcd6-df9b30b59879 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4814f42888d4554a082b2bdb2036ac8', 'request_id': 'req-599abaa5-a14c-4a43-bcd6-df9b30b59879'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:26:41.431726 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-599abaa5-a14c-4a43-bcd6-df9b30b59879 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4814f42888d4554a082b2bdb2036ac8', 'request_id': 'req-599abaa5-a14c-4a43-bcd6-df9b30b59879'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:26:41.437824 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-599abaa5-a14c-4a43-bcd6-df9b30b59879 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4814f42888d4554a082b2bdb2036ac8', 'request_id': 'req-599abaa5-a14c-4a43-bcd6-df9b30b59879'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:26:41.449279 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-599abaa5-a14c-4a43-bcd6-df9b30b59879 admin admin] The token cache is being invalidate because project f4814f42888d4554a082b2bdb2036ac8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86933) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:26:41.449602 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-599abaa5-a14c-4a43-bcd6-df9b30b59879 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:26:41.450252 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-599abaa5-a14c-4a43-bcd6-df9b30b59879 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:26:41.451773 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1412/2813] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 14:26:41 2022] DELETE /identity/v3/projects/f4814f42888d4554a082b2bdb2036ac8 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:26:41.458157 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c132f2b5-f696-42fd-af56-04acf2cb2fd0 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:26:41.458446 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c132f2b5-f696-42fd-af56-04acf2cb2fd0 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:26:41.458719 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c132f2b5-f696-42fd-af56-04acf2cb2fd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:26:41.478288 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-c132f2b5-f696-42fd-af56-04acf2cb2fd0 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:26:41.479482 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-c132f2b5-f696-42fd-af56-04acf2cb2fd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:26:41.517721 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:26:41.539808 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1402/2814] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:26:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:26:41.549015 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e77e16-28a9-4ae3-9ef3-0c5e41879193 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:26:41.549299 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e77e16-28a9-4ae3-9ef3-0c5e41879193 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:26:41.549613 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e77e16-28a9-4ae3-9ef3-0c5e41879193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:26:41.563834 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:26:41.572568 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-57e77e16-28a9-4ae3-9ef3-0c5e41879193 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:26:41.574416 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-57e77e16-28a9-4ae3-9ef3-0c5e41879193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:26:41.583320 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1413/2815] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:26:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:26:41.587508 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c132f2b5-f696-42fd-af56-04acf2cb2fd0 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:26:41.597910 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c132f2b5-f696-42fd-af56-04acf2cb2fd0 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:26:41.602595 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839031f4-9253-44ee-b2a9-bbfbd34ef480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:26:41.603202 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839031f4-9253-44ee-b2a9-bbfbd34ef480 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:26:41.603481 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839031f4-9253-44ee-b2a9-bbfbd34ef480 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:26:41.603750 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839031f4-9253-44ee-b2a9-bbfbd34ef480 admin admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:26:41.604438 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839031f4-9253-44ee-b2a9-bbfbd34ef480 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:26:41.604739 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839031f4-9253-44ee-b2a9-bbfbd34ef480 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:26:41.606347 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839031f4-9253-44ee-b2a9-bbfbd34ef480 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:26:41.617751 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1403/2816] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Jul 26 14:26:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:26:41.622104 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e77e16-28a9-4ae3-9ef3-0c5e41879193 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:26:41.631951 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e77e16-28a9-4ae3-9ef3-0c5e41879193 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:26:41.636599 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1936201-6d71-4e7f-99ea-31986ef859fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:26:41.637200 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1936201-6d71-4e7f-99ea-31986ef859fe admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:26:41.637501 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1936201-6d71-4e7f-99ea-31986ef859fe admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:26:41.637783 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1936201-6d71-4e7f-99ea-31986ef859fe admin admin] PATH_INFO: `/v3/projects` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:26:41.638447 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1936201-6d71-4e7f-99ea-31986ef859fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:26:41.640952 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1936201-6d71-4e7f-99ea-31986ef859fe admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:26:41.668122 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1414/2817] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 14:26:41 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:26:41.672671 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839031f4-9253-44ee-b2a9-bbfbd34ef480 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:26:41.682657 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839031f4-9253-44ee-b2a9-bbfbd34ef480 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:26:41.741257 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2854e1-3a0f-49df-85f9-fa0bec970c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:26:41.741911 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2854e1-3a0f-49df-85f9-fa0bec970c03 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:26:41.742194 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2854e1-3a0f-49df-85f9-fa0bec970c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:26:41.742469 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2854e1-3a0f-49df-85f9-fa0bec970c03 admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:26:41.743128 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2854e1-3a0f-49df-85f9-fa0bec970c03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:26:41.745028 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2854e1-3a0f-49df-85f9-fa0bec970c03 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:26:41.770569 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1404/2818] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 14:26:41 2022] POST /identity/v3/users => generated 289 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:26:41.774601 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1936201-6d71-4e7f-99ea-31986ef859fe admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:26:41.784796 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1936201-6d71-4e7f-99ea-31986ef859fe admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:26:41.789526 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9413ab78-a299-4623-a7ea-b077c23bc234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:26:41.790153 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9413ab78-a299-4623-a7ea-b077c23bc234 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:26:41.790492 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9413ab78-a299-4623-a7ea-b077c23bc234 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:26:41.790687 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9413ab78-a299-4623-a7ea-b077c23bc234 admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:26:41.791420 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9413ab78-a299-4623-a7ea-b077c23bc234 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:26:41.792966 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9413ab78-a299-4623-a7ea-b077c23bc234 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:26:41.804207 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1415/2819] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:26:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:26:41.808534 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2854e1-3a0f-49df-85f9-fa0bec970c03 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:26:41.820197 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2854e1-3a0f-49df-85f9-fa0bec970c03 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:26:41.824877 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d40dfb7-10d4-4a86-b4f3-f8cecb1eb37a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:26:41.825671 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d40dfb7-10d4-4a86-b4f3-f8cecb1eb37a admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:26:41.825987 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d40dfb7-10d4-4a86-b4f3-f8cecb1eb37a admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:26:41.826272 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d40dfb7-10d4-4a86-b4f3-f8cecb1eb37a admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:26:41.826974 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d40dfb7-10d4-4a86-b4f3-f8cecb1eb37a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:26:41.829223 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d40dfb7-10d4-4a86-b4f3-f8cecb1eb37a admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:26:41.839978 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1405/2820] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:26:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:26:41.844375 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9413ab78-a299-4623-a7ea-b077c23bc234 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:26:41.854297 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9413ab78-a299-4623-a7ea-b077c23bc234 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:26:41.859878 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6674e94-4aba-4abc-9bed-e96ba0df65a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:26:41.860539 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6674e94-4aba-4abc-9bed-e96ba0df65a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:26:41.860818 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6674e94-4aba-4abc-9bed-e96ba0df65a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:26:41.861155 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6674e94-4aba-4abc-9bed-e96ba0df65a5 admin admin] PATH_INFO: `/v3/projects/7cddd953364c488994815a0585d9be72/users/24bd5eb6e7fd46ebb95fbc72ea7f40c3/roles/759f389b221049c4b01faf3b984cd8d7` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:26:41.876141 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6674e94-4aba-4abc-9bed-e96ba0df65a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cddd953364c488994815a0585d9be72, user_id=24bd5eb6e7fd46ebb95fbc72ea7f40c3, role_id=759f389b221049c4b01faf3b984cd8d7)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:26:41.878779 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6674e94-4aba-4abc-9bed-e96ba0df65a5 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:26:41.891971 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1416/2821] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 14:26:41 2022] PUT /identity/v3/projects/7cddd953364c488994815a0585d9be72/users/24bd5eb6e7fd46ebb95fbc72ea7f40c3/roles/759f389b221049c4b01faf3b984cd8d7 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:26:41.901952 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139a0152-abc7-4982-aca7-f36ac5b219a6 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:26:41.902271 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139a0152-abc7-4982-aca7-f36ac5b219a6 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:26:41.902814 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139a0152-abc7-4982-aca7-f36ac5b219a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:26:41.931563 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-139a0152-abc7-4982-aca7-f36ac5b219a6 None None] MFA Rules not processed for user `24bd5eb6e7fd46ebb95fbc72ea7f40c3`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:26:41.932809 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-139a0152-abc7-4982-aca7-f36ac5b219a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:26:41.996434 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1406/2822] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:26:41 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:26:42.001598 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6674e94-4aba-4abc-9bed-e96ba0df65a5 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:26:42.063544 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6674e94-4aba-4abc-9bed-e96ba0df65a5 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:26:42.068170 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c4bbb7-68e0-4484-baff-474d089c18a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:26:42.068779 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c4bbb7-68e0-4484-baff-474d089c18a2 admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:26:42.069058 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c4bbb7-68e0-4484-baff-474d089c18a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:26:42.069328 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c4bbb7-68e0-4484-baff-474d089c18a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:26:42.070272 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c4bbb7-68e0-4484-baff-474d089c18a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:26:42.072147 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c4bbb7-68e0-4484-baff-474d089c18a2 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:26:42.097878 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1417/2823] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 14:26:42 2022] POST /identity/v3/projects => generated 381 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:26:42.101998 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139a0152-abc7-4982-aca7-f36ac5b219a6 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:26:42.113110 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139a0152-abc7-4982-aca7-f36ac5b219a6 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:26:42.145865 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:26:42.168657 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12da63a9-7ec7-4704-8910-9dc48424e35c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:26:42.169273 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12da63a9-7ec7-4704-8910-9dc48424e35c admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:26:42.169588 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12da63a9-7ec7-4704-8910-9dc48424e35c admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:26:42.169862 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12da63a9-7ec7-4704-8910-9dc48424e35c admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:26:42.170512 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12da63a9-7ec7-4704-8910-9dc48424e35c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:26:42.173031 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12da63a9-7ec7-4704-8910-9dc48424e35c admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:26:42.195125 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1407/2824] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 14:26:42 2022] POST /identity/v3/users => generated 297 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:26:42.199078 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c4bbb7-68e0-4484-baff-474d089c18a2 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:26:42.209688 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c4bbb7-68e0-4484-baff-474d089c18a2 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:26:42.214233 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a46db84-d1c9-4c7f-a5b9-d1aa2136df42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:26:42.214838 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a46db84-d1c9-4c7f-a5b9-d1aa2136df42 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:26:42.215119 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a46db84-d1c9-4c7f-a5b9-d1aa2136df42 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:26:42.215397 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a46db84-d1c9-4c7f-a5b9-d1aa2136df42 admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:26:42.216078 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a46db84-d1c9-4c7f-a5b9-d1aa2136df42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:26:42.217682 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a46db84-d1c9-4c7f-a5b9-d1aa2136df42 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:26:42.228573 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1418/2825] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:26:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:26:42.232761 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12da63a9-7ec7-4704-8910-9dc48424e35c admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:26:42.243201 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12da63a9-7ec7-4704-8910-9dc48424e35c admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:26:42.247742 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a5fff2-bf76-4d4e-8ab0-937cb938b44f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:26:42.248393 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a5fff2-bf76-4d4e-8ab0-937cb938b44f admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:26:42.248673 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a5fff2-bf76-4d4e-8ab0-937cb938b44f admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:26:42.248949 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a5fff2-bf76-4d4e-8ab0-937cb938b44f admin admin] PATH_INFO: `/v3/projects/8ab39b866f3d4cabad9b0622ee38b298/users/9ab7f3e664b1485b967ecf312f01ea8b/roles/6e0201a9a6484f7d8b112b1eafff70bd` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:26:42.263767 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a5fff2-bf76-4d4e-8ab0-937cb938b44f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ab39b866f3d4cabad9b0622ee38b298, user_id=9ab7f3e664b1485b967ecf312f01ea8b, role_id=6e0201a9a6484f7d8b112b1eafff70bd)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:26:42.266571 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a5fff2-bf76-4d4e-8ab0-937cb938b44f admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:26:42.281161 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1408/2826] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 14:26:42 2022] PUT /identity/v3/projects/8ab39b866f3d4cabad9b0622ee38b298/users/9ab7f3e664b1485b967ecf312f01ea8b/roles/6e0201a9a6484f7d8b112b1eafff70bd => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:26:42.285080 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a46db84-d1c9-4c7f-a5b9-d1aa2136df42 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:26:42.295537 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a46db84-d1c9-4c7f-a5b9-d1aa2136df42 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:26:42.354088 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d62e5a0-3a34-4c1b-b225-b7e3a9af2d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:26:42.354737 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d62e5a0-3a34-4c1b-b225-b7e3a9af2d00 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:26:42.355043 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d62e5a0-3a34-4c1b-b225-b7e3a9af2d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:26:42.355320 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d62e5a0-3a34-4c1b-b225-b7e3a9af2d00 admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:26:42.356022 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d62e5a0-3a34-4c1b-b225-b7e3a9af2d00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:26:42.357654 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d62e5a0-3a34-4c1b-b225-b7e3a9af2d00 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:26:42.368258 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1419/2827] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:26:42 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:26:42.372376 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a5fff2-bf76-4d4e-8ab0-937cb938b44f admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:26:42.384187 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a5fff2-bf76-4d4e-8ab0-937cb938b44f admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:26:42.389358 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aedad1-aa69-498c-99cd-e6f1062ab484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:26:42.390149 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aedad1-aa69-498c-99cd-e6f1062ab484 admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:26:42.390434 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aedad1-aa69-498c-99cd-e6f1062ab484 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:26:42.390713 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aedad1-aa69-498c-99cd-e6f1062ab484 admin admin] PATH_INFO: `/v3/projects/8ab39b866f3d4cabad9b0622ee38b298/users/9ab7f3e664b1485b967ecf312f01ea8b/roles/6e0201a9a6484f7d8b112b1eafff70bd` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:26:42.392811 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10aedad1-aa69-498c-99cd-e6f1062ab484 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ab39b866f3d4cabad9b0622ee38b298, user_id=9ab7f3e664b1485b967ecf312f01ea8b, role_id=6e0201a9a6484f7d8b112b1eafff70bd)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:26:42.395527 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10aedad1-aa69-498c-99cd-e6f1062ab484 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:26:42.406573 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1409/2828] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 14:26:42 2022] PUT /identity/v3/projects/8ab39b866f3d4cabad9b0622ee38b298/users/9ab7f3e664b1485b967ecf312f01ea8b/roles/6e0201a9a6484f7d8b112b1eafff70bd => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:26:42.417039 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b27388-efc2-4633-a2b3-7579c51d401d None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:26:42.417331 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b27388-efc2-4633-a2b3-7579c51d401d None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:26:42.417675 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b27388-efc2-4633-a2b3-7579c51d401d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:26:42.444038 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:26:42.448543 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-c0b27388-efc2-4633-a2b3-7579c51d401d None None] MFA Rules not processed for user `9ab7f3e664b1485b967ecf312f01ea8b`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:26:42.451072 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-c0b27388-efc2-4633-a2b3-7579c51d401d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:26:42.492936 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e02e26-c71a-4680-a54b-adc9e07734d1 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:26:42.493230 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e02e26-c71a-4680-a54b-adc9e07734d1 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:26:42.493536 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e02e26-c71a-4680-a54b-adc9e07734d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:26:42.518216 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-71e02e26-c71a-4680-a54b-adc9e07734d1 None None] MFA Rules not processed for user `b64bd7b183d24d8c9037b261c7884810`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:26:42.519695 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-71e02e26-c71a-4680-a54b-adc9e07734d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:26:42.536981 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1420/2829] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:26:42 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:26:42.549094 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b27388-efc2-4633-a2b3-7579c51d401d None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:26:42.550468 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-c0b27388-efc2-4633-a2b3-7579c51d401d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:26:42.586306 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1410/2830] 10.222.0.58 () {58 vars in 1009 bytes} [Tue Jul 26 14:26:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:26:42.611102 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b27388-efc2-4633-a2b3-7579c51d401d None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:26:42.614727 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ab825c-9250-4060-9673-e3242874cde5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:26:42.615355 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ab825c-9250-4060-9673-e3242874cde5 service cinder] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:26:42.615617 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ab825c-9250-4060-9673-e3242874cde5 service cinder] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:26:42.615894 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ab825c-9250-4060-9673-e3242874cde5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:26:42.625504 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ab825c-9250-4060-9673-e3242874cde5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:26:42.626933 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ab825c-9250-4060-9673-e3242874cde5 service cinder] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:26:42.637019 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e02e26-c71a-4680-a54b-adc9e07734d1 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:26:42.637571 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1421/2831] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:26:42 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 14:26:42.638468 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-71e02e26-c71a-4680-a54b-adc9e07734d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:26:42.714790 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ab825c-9250-4060-9673-e3242874cde5 service cinder] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:26:42.715340 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e02e26-c71a-4680-a54b-adc9e07734d1 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:26:42.721566 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a274b24c-00ac-492e-ad51-d4b7569024f0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab2819bac844163ae4305e4f2186605', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:26:42.722201 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a274b24c-00ac-492e-ad51-d4b7569024f0 service swift] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:26:42.722492 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a274b24c-00ac-492e-ad51-d4b7569024f0 service swift] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:26:42.722773 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a274b24c-00ac-492e-ad51-d4b7569024f0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:26:42.724308 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ab825c-9250-4060-9673-e3242874cde5 service cinder] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:26:42.727976 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a0cc4c-c196-439a-ad66-8a22db0a7c30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:26:42.728578 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a0cc4c-c196-439a-ad66-8a22db0a7c30 service cinder] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:26:42.728854 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a0cc4c-c196-439a-ad66-8a22db0a7c30 service cinder] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:26:42.729127 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a0cc4c-c196-439a-ad66-8a22db0a7c30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:26:42.731773 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a274b24c-00ac-492e-ad51-d4b7569024f0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:26:42.733374 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a274b24c-00ac-492e-ad51-d4b7569024f0 service swift] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:26:42.741750 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1411/2832] 10.222.0.58 () {60 vars in 1384 bytes} [Tue Jul 26 14:26:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 26 14:26:42.799781 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a0cc4c-c196-439a-ad66-8a22db0a7c30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:26:42.802205 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a0cc4c-c196-439a-ad66-8a22db0a7c30 service cinder] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:26:42.814327 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1422/2833] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:26:42 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 14:26:52.738486 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=87,keystone:INSERT=5 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:26:52.809158 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=82 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:26:54.714948 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a274b24c-00ac-492e-ad51-d4b7569024f0 service swift] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:26:54.863226 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a274b24c-00ac-492e-ad51-d4b7569024f0 service swift] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:26:54.868894 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2f69ab-7373-498b-aa93-01aea8a651fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:26:54.869707 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2f69ab-7373-498b-aa93-01aea8a651fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:26:54.869929 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2f69ab-7373-498b-aa93-01aea8a651fa admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:26:54.870509 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2f69ab-7373-498b-aa93-01aea8a651fa admin admin] PATH_INFO: `/v3/users/24bd5eb6e7fd46ebb95fbc72ea7f40c3` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:26:54.872045 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2f69ab-7373-498b-aa93-01aea8a651fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=24bd5eb6e7fd46ebb95fbc72ea7f40c3)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:26:54.875954 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2f69ab-7373-498b-aa93-01aea8a651fa admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:26:54.975360 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-8c2f69ab-7373-498b-aa93-01aea8a651fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24bd5eb6e7fd46ebb95fbc72ea7f40c3'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:26:54.976397 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-8c2f69ab-7373-498b-aa93-01aea8a651fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24bd5eb6e7fd46ebb95fbc72ea7f40c3'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:26:55.001502 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-8c2f69ab-7373-498b-aa93-01aea8a651fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24bd5eb6e7fd46ebb95fbc72ea7f40c3'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:26:55.004049 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-8c2f69ab-7373-498b-aa93-01aea8a651fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24bd5eb6e7fd46ebb95fbc72ea7f40c3'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:26:55.048692 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-8c2f69ab-7373-498b-aa93-01aea8a651fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24bd5eb6e7fd46ebb95fbc72ea7f40c3'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:26:55.088545 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1412/2834] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 14:26:54 2022] DELETE /identity/v3/users/24bd5eb6e7fd46ebb95fbc72ea7f40c3 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:26:55.093531 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63620726-6f77-4709-8036-5f2639b2fa1c None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:26:55.094353 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63620726-6f77-4709-8036-5f2639b2fa1c None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:26:55.094742 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63620726-6f77-4709-8036-5f2639b2fa1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:26:55.135641 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-63620726-6f77-4709-8036-5f2639b2fa1c None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:26:55.139320 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-63620726-6f77-4709-8036-5f2639b2fa1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:26:55.250147 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1423/2835] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:26:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:26:55.261018 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2f69ab-7373-498b-aa93-01aea8a651fa admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:26:55.263590 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-8c2f69ab-7373-498b-aa93-01aea8a651fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:26:55.384740 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2f69ab-7373-498b-aa93-01aea8a651fa admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:26:55.390747 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456e2be9-621f-4707-ad8e-95672c1c3759 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:26:55.391406 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456e2be9-621f-4707-ad8e-95672c1c3759 service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:26:55.391787 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456e2be9-621f-4707-ad8e-95672c1c3759 service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:26:55.392132 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456e2be9-621f-4707-ad8e-95672c1c3759 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:26:55.404648 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456e2be9-621f-4707-ad8e-95672c1c3759 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:26:55.406500 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456e2be9-621f-4707-ad8e-95672c1c3759 service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:26:55.417955 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1413/2836] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:26:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:26:55.881778 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63620726-6f77-4709-8036-5f2639b2fa1c None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:26:55.883997 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-63620726-6f77-4709-8036-5f2639b2fa1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:26:55.911497 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63620726-6f77-4709-8036-5f2639b2fa1c None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:26:55.924565 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da182d78-c97f-4e8a-9b72-611898c52ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:26:55.925827 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da182d78-c97f-4e8a-9b72-611898c52ac1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:26:55.926349 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da182d78-c97f-4e8a-9b72-611898c52ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:26:55.926859 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da182d78-c97f-4e8a-9b72-611898c52ac1 admin admin] PATH_INFO: `/v3/projects/7cddd953364c488994815a0585d9be72` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:26:55.934424 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da182d78-c97f-4e8a-9b72-611898c52ac1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7cddd953364c488994815a0585d9be72)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:26:55.936649 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da182d78-c97f-4e8a-9b72-611898c52ac1 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:26:56.004739 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-da182d78-c97f-4e8a-9b72-611898c52ac1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7cddd953364c488994815a0585d9be72', 'request_id': 'req-da182d78-c97f-4e8a-9b72-611898c52ac1'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:26:56.020253 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-da182d78-c97f-4e8a-9b72-611898c52ac1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7cddd953364c488994815a0585d9be72', 'request_id': 'req-da182d78-c97f-4e8a-9b72-611898c52ac1'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:26:56.037525 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-da182d78-c97f-4e8a-9b72-611898c52ac1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7cddd953364c488994815a0585d9be72', 'request_id': 'req-da182d78-c97f-4e8a-9b72-611898c52ac1'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:26:56.057368 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-da182d78-c97f-4e8a-9b72-611898c52ac1 admin admin] The token cache is being invalidate because project 7cddd953364c488994815a0585d9be72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86933) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:26:56.057849 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-da182d78-c97f-4e8a-9b72-611898c52ac1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:26:56.058540 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-da182d78-c97f-4e8a-9b72-611898c52ac1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:26:56.060256 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1424/2837] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 14:26:55 2022] DELETE /identity/v3/projects/7cddd953364c488994815a0585d9be72 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:26:56.064356 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456e2be9-621f-4707-ad8e-95672c1c3759 service neutron] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:26:56.070126 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:26:56.082010 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456e2be9-621f-4707-ad8e-95672c1c3759 service neutron] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:26:56.205138 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60606aa0-ae0c-408d-a166-376302af4fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:26:56.205907 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60606aa0-ae0c-408d-a166-376302af4fd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:26:56.206199 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60606aa0-ae0c-408d-a166-376302af4fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:26:56.206567 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60606aa0-ae0c-408d-a166-376302af4fd5 admin admin] PATH_INFO: `/v3/users/9ab7f3e664b1485b967ecf312f01ea8b` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:26:56.208102 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60606aa0-ae0c-408d-a166-376302af4fd5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ab7f3e664b1485b967ecf312f01ea8b)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:26:56.210529 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60606aa0-ae0c-408d-a166-376302af4fd5 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:26:56.309461 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-60606aa0-ae0c-408d-a166-376302af4fd5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ab7f3e664b1485b967ecf312f01ea8b'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:26:56.310538 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-60606aa0-ae0c-408d-a166-376302af4fd5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ab7f3e664b1485b967ecf312f01ea8b'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:26:56.325450 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-60606aa0-ae0c-408d-a166-376302af4fd5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ab7f3e664b1485b967ecf312f01ea8b'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:26:56.327660 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-60606aa0-ae0c-408d-a166-376302af4fd5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ab7f3e664b1485b967ecf312f01ea8b'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:26:56.350802 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-60606aa0-ae0c-408d-a166-376302af4fd5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ab7f3e664b1485b967ecf312f01ea8b'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:26:56.373533 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1414/2838] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 14:26:56 2022] DELETE /identity/v3/users/9ab7f3e664b1485b967ecf312f01ea8b => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:26:56.843448 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da182d78-c97f-4e8a-9b72-611898c52ac1 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:26:56.846899 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-da182d78-c97f-4e8a-9b72-611898c52ac1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:26:56.864938 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da182d78-c97f-4e8a-9b72-611898c52ac1 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:26:56.961360 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6025988-028e-471f-9998-1cc9e97c4aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:26:56.962337 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6025988-028e-471f-9998-1cc9e97c4aef admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:26:56.962690 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6025988-028e-471f-9998-1cc9e97c4aef admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:26:56.963506 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6025988-028e-471f-9998-1cc9e97c4aef admin admin] PATH_INFO: `/v3/projects/8ab39b866f3d4cabad9b0622ee38b298` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:26:56.964396 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6025988-028e-471f-9998-1cc9e97c4aef admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ab39b866f3d4cabad9b0622ee38b298)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:26:56.966344 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6025988-028e-471f-9998-1cc9e97c4aef admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:26:57.011163 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:26:57.055033 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-a6025988-028e-471f-9998-1cc9e97c4aef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ab39b866f3d4cabad9b0622ee38b298', 'request_id': 'req-a6025988-028e-471f-9998-1cc9e97c4aef'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:26:57.064521 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-a6025988-028e-471f-9998-1cc9e97c4aef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ab39b866f3d4cabad9b0622ee38b298', 'request_id': 'req-a6025988-028e-471f-9998-1cc9e97c4aef'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:26:57.074499 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-a6025988-028e-471f-9998-1cc9e97c4aef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ab39b866f3d4cabad9b0622ee38b298', 'request_id': 'req-a6025988-028e-471f-9998-1cc9e97c4aef'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:26:57.090366 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-a6025988-028e-471f-9998-1cc9e97c4aef admin admin] The token cache is being invalidate because project 8ab39b866f3d4cabad9b0622ee38b298 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86933) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:26:57.090664 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-a6025988-028e-471f-9998-1cc9e97c4aef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:26:57.091716 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-a6025988-028e-471f-9998-1cc9e97c4aef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:26:57.094239 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1425/2839] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 14:26:56 2022] DELETE /identity/v3/projects/8ab39b866f3d4cabad9b0622ee38b298 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:26:57.100580 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b956e96-1403-4114-8d13-5462a68da04a None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:26:57.101802 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b956e96-1403-4114-8d13-5462a68da04a None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:26:57.101802 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b956e96-1403-4114-8d13-5462a68da04a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:26:57.127839 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-8b956e96-1403-4114-8d13-5462a68da04a None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:26:57.129898 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-8b956e96-1403-4114-8d13-5462a68da04a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:26:57.226318 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1415/2840] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:26:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:26:57.235790 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c4e207-eb55-4e9b-bb9f-11373378203e None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:26:57.236151 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c4e207-eb55-4e9b-bb9f-11373378203e None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:26:57.236486 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c4e207-eb55-4e9b-bb9f-11373378203e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:26:57.262280 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-50c4e207-eb55-4e9b-bb9f-11373378203e None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:26:57.263601 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-50c4e207-eb55-4e9b-bb9f-11373378203e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:26:57.275942 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1426/2841] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:26:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:26:57.280881 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b956e96-1403-4114-8d13-5462a68da04a None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:26:57.293916 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b956e96-1403-4114-8d13-5462a68da04a None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:26:57.300132 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9279f6-df8c-4047-a97c-bfcd4f2050d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:26:57.300847 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9279f6-df8c-4047-a97c-bfcd4f2050d7 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:26:57.301204 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9279f6-df8c-4047-a97c-bfcd4f2050d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:26:57.301586 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9279f6-df8c-4047-a97c-bfcd4f2050d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:26:57.302387 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9279f6-df8c-4047-a97c-bfcd4f2050d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:26:57.302754 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9279f6-df8c-4047-a97c-bfcd4f2050d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:26:57.304681 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9279f6-df8c-4047-a97c-bfcd4f2050d7 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:26:57.324394 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1416/2842] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Jul 26 14:26:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:26:57.329822 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c4e207-eb55-4e9b-bb9f-11373378203e None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:26:57.340074 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c4e207-eb55-4e9b-bb9f-11373378203e None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:26:57.347351 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55c6e31-a085-4ad1-a646-e12428b6fd6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:26:57.348070 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55c6e31-a085-4ad1-a646-e12428b6fd6d admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:26:57.348466 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55c6e31-a085-4ad1-a646-e12428b6fd6d admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:26:57.348789 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55c6e31-a085-4ad1-a646-e12428b6fd6d admin admin] PATH_INFO: `/v3/projects` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:26:57.349780 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55c6e31-a085-4ad1-a646-e12428b6fd6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:26:57.351753 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55c6e31-a085-4ad1-a646-e12428b6fd6d admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:26:57.386366 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1427/2843] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 14:26:57 2022] POST /identity/v3/projects => generated 375 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:26:57.391934 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9279f6-df8c-4047-a97c-bfcd4f2050d7 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:26:57.408629 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9279f6-df8c-4047-a97c-bfcd4f2050d7 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:26:57.416815 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:26:57.521744 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31da261-614e-4ba8-8ca5-3c202f5dc1bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:26:57.523261 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31da261-614e-4ba8-8ca5-3c202f5dc1bb admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:26:57.523806 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31da261-614e-4ba8-8ca5-3c202f5dc1bb admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:26:57.524353 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31da261-614e-4ba8-8ca5-3c202f5dc1bb admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:26:57.525645 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31da261-614e-4ba8-8ca5-3c202f5dc1bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:26:57.528826 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31da261-614e-4ba8-8ca5-3c202f5dc1bb admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:26:57.558716 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1417/2844] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 14:26:57 2022] POST /identity/v3/users => generated 288 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:26:57.564343 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55c6e31-a085-4ad1-a646-e12428b6fd6d admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:26:57.581164 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55c6e31-a085-4ad1-a646-e12428b6fd6d admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:26:57.591800 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c68821c-18b1-4783-9996-5ac88684455c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:26:57.593394 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c68821c-18b1-4783-9996-5ac88684455c admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:26:57.594208 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c68821c-18b1-4783-9996-5ac88684455c admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:26:57.594975 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c68821c-18b1-4783-9996-5ac88684455c admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:26:57.596374 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c68821c-18b1-4783-9996-5ac88684455c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:26:57.599852 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c68821c-18b1-4783-9996-5ac88684455c admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:26:57.626855 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1428/2845] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:26:57 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:26:57.628116 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31da261-614e-4ba8-8ca5-3c202f5dc1bb admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:26:57.647953 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31da261-614e-4ba8-8ca5-3c202f5dc1bb admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:26:57.658206 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f38c858-98c7-410b-ac49-7878de084b16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:26:57.659473 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f38c858-98c7-410b-ac49-7878de084b16 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:26:57.659873 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f38c858-98c7-410b-ac49-7878de084b16 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:26:57.660224 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f38c858-98c7-410b-ac49-7878de084b16 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:26:57.661023 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f38c858-98c7-410b-ac49-7878de084b16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:26:57.663847 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f38c858-98c7-410b-ac49-7878de084b16 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:26:57.681062 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1418/2846] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:26:57 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:26:57.686778 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c68821c-18b1-4783-9996-5ac88684455c admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:26:57.701299 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c68821c-18b1-4783-9996-5ac88684455c admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:26:57.712782 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcfb3cc-7de2-4fc0-9057-6b439fe3c18a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:26:57.714092 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcfb3cc-7de2-4fc0-9057-6b439fe3c18a admin admin] REQUEST_METHOD: `PUT` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:26:57.714621 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcfb3cc-7de2-4fc0-9057-6b439fe3c18a admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:26:57.715157 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcfb3cc-7de2-4fc0-9057-6b439fe3c18a admin admin] PATH_INFO: `/v3/projects/d4c0f9d5d84a4ed2aa6368c212c8f3e1/users/fa533a8124b64affa12b7d624c778da7/roles/759f389b221049c4b01faf3b984cd8d7` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:26:57.744469 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcfb3cc-7de2-4fc0-9057-6b439fe3c18a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4c0f9d5d84a4ed2aa6368c212c8f3e1, user_id=fa533a8124b64affa12b7d624c778da7, role_id=759f389b221049c4b01faf3b984cd8d7)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:26:57.746175 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcfb3cc-7de2-4fc0-9057-6b439fe3c18a admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:26:57.770744 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1429/2847] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 14:26:57 2022] PUT /identity/v3/projects/d4c0f9d5d84a4ed2aa6368c212c8f3e1/users/fa533a8124b64affa12b7d624c778da7/roles/759f389b221049c4b01faf3b984cd8d7 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:26:57.786092 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3ea250-d444-4b2f-acfa-0b412dcc4313 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:26:57.786641 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3ea250-d444-4b2f-acfa-0b412dcc4313 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:26:57.787172 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3ea250-d444-4b2f-acfa-0b412dcc4313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:26:57.835094 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-fc3ea250-d444-4b2f-acfa-0b412dcc4313 None None] MFA Rules not processed for user `fa533a8124b64affa12b7d624c778da7`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:26:57.837997 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-fc3ea250-d444-4b2f-acfa-0b412dcc4313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:26:58.013531 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1419/2848] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:26:57 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:26:58.030182 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcfb3cc-7de2-4fc0-9057-6b439fe3c18a admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:26:58.034812 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-bdcfb3cc-7de2-4fc0-9057-6b439fe3c18a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:26:58.147790 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcfb3cc-7de2-4fc0-9057-6b439fe3c18a admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:26:58.154377 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81850cad-a0e5-4cf4-a0a7-c2c7baa19ed0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf065aef9a046b2adaec34816a336b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:26:58.156285 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81850cad-a0e5-4cf4-a0a7-c2c7baa19ed0 service glance] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:26:58.156846 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81850cad-a0e5-4cf4-a0a7-c2c7baa19ed0 service glance] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:26:58.157445 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81850cad-a0e5-4cf4-a0a7-c2c7baa19ed0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:26:58.174596 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81850cad-a0e5-4cf4-a0a7-c2c7baa19ed0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:26:58.177287 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81850cad-a0e5-4cf4-a0a7-c2c7baa19ed0 service glance] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:26:58.194422 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1430/2849] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:26:58 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:27:01.976806 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ee8a00-3eee-4062-82c2-a1e3a8012eb9 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:27:01.977326 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ee8a00-3eee-4062-82c2-a1e3a8012eb9 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:27:01.977326 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ee8a00-3eee-4062-82c2-a1e3a8012eb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:27:01.997761 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-95ee8a00-3eee-4062-82c2-a1e3a8012eb9 None None] MFA Rules not processed for user `b64bd7b183d24d8c9037b261c7884810`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:27:01.999070 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-95ee8a00-3eee-4062-82c2-a1e3a8012eb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:27:02.062506 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1420/2850] 10.222.0.58 () {58 vars in 967 bytes} [Tue Jul 26 14:27:01 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:27:02.075735 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81850cad-a0e5-4cf4-a0a7-c2c7baa19ed0 service glance] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:27:02.077717 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-81850cad-a0e5-4cf4-a0a7-c2c7baa19ed0 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:27:02.130620 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:27:02.149709 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81850cad-a0e5-4cf4-a0a7-c2c7baa19ed0 service glance] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:27:02.155782 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3dd8a0-f323-4b84-8edd-f48b5ce28fbe service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ab2819bac844163ae4305e4f2186605', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:27:02.156410 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3dd8a0-f323-4b84-8edd-f48b5ce28fbe service swift] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:27:02.156685 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3dd8a0-f323-4b84-8edd-f48b5ce28fbe service swift] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:27:02.156958 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3dd8a0-f323-4b84-8edd-f48b5ce28fbe service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:27:02.165812 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3dd8a0-f323-4b84-8edd-f48b5ce28fbe service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:27:02.167417 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3dd8a0-f323-4b84-8edd-f48b5ce28fbe service swift] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:27:02.176192 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1431/2851] 10.222.0.58 () {60 vars in 1384 bytes} [Tue Jul 26 14:27:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 14:27:04.171482 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ee8a00-3eee-4062-82c2-a1e3a8012eb9 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:27:04.174652 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-95ee8a00-3eee-4062-82c2-a1e3a8012eb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:27:04.205972 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:27:04.249502 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ee8a00-3eee-4062-82c2-a1e3a8012eb9 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:27:04.254162 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad525ea4-d0cc-4e9b-8558-db48922b0b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:27:04.254813 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad525ea4-d0cc-4e9b-8558-db48922b0b33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:27:04.255101 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad525ea4-d0cc-4e9b-8558-db48922b0b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:27:04.255383 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad525ea4-d0cc-4e9b-8558-db48922b0b33 admin admin] PATH_INFO: `/v3/users/338a1a2e73d84021a91b7a90e8b83b7d` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:27:04.256813 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad525ea4-d0cc-4e9b-8558-db48922b0b33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=338a1a2e73d84021a91b7a90e8b83b7d)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:27:04.258779 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad525ea4-d0cc-4e9b-8558-db48922b0b33 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:27:04.312061 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-ad525ea4-d0cc-4e9b-8558-db48922b0b33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '338a1a2e73d84021a91b7a90e8b83b7d'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:27:04.312920 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-ad525ea4-d0cc-4e9b-8558-db48922b0b33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '338a1a2e73d84021a91b7a90e8b83b7d'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:27:04.320846 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-ad525ea4-d0cc-4e9b-8558-db48922b0b33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '338a1a2e73d84021a91b7a90e8b83b7d'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:27:04.321591 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-ad525ea4-d0cc-4e9b-8558-db48922b0b33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '338a1a2e73d84021a91b7a90e8b83b7d'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:27:04.334204 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-ad525ea4-d0cc-4e9b-8558-db48922b0b33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '338a1a2e73d84021a91b7a90e8b83b7d'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:27:04.347634 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1421/2852] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 14:27:04 2022] DELETE /identity/v3/users/338a1a2e73d84021a91b7a90e8b83b7d => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:27:04.541296 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3dd8a0-f323-4b84-8edd-f48b5ce28fbe service swift] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:27:04.552973 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3dd8a0-f323-4b84-8edd-f48b5ce28fbe service swift] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:27:04.616525 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c7f1a5-4d84-491c-aae1-a91fa310d141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:27:04.617191 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c7f1a5-4d84-491c-aae1-a91fa310d141 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:27:04.617588 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c7f1a5-4d84-491c-aae1-a91fa310d141 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:27:04.617936 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c7f1a5-4d84-491c-aae1-a91fa310d141 admin admin] PATH_INFO: `/v3/projects/b78358f25fbf41569b73f7500baee3cc` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:27:04.619202 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c7f1a5-4d84-491c-aae1-a91fa310d141 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b78358f25fbf41569b73f7500baee3cc)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:27:04.621128 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c7f1a5-4d84-491c-aae1-a91fa310d141 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:27:04.668881 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-e3c7f1a5-4d84-491c-aae1-a91fa310d141 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b78358f25fbf41569b73f7500baee3cc', 'request_id': 'req-e3c7f1a5-4d84-491c-aae1-a91fa310d141'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:27:04.677519 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-e3c7f1a5-4d84-491c-aae1-a91fa310d141 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b78358f25fbf41569b73f7500baee3cc', 'request_id': 'req-e3c7f1a5-4d84-491c-aae1-a91fa310d141'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:27:04.683671 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-e3c7f1a5-4d84-491c-aae1-a91fa310d141 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b78358f25fbf41569b73f7500baee3cc', 'request_id': 'req-e3c7f1a5-4d84-491c-aae1-a91fa310d141'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:27:04.696657 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-e3c7f1a5-4d84-491c-aae1-a91fa310d141 admin admin] The token cache is being invalidate because project b78358f25fbf41569b73f7500baee3cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86933) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:27:04.696946 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-e3c7f1a5-4d84-491c-aae1-a91fa310d141 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:27:04.697637 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-e3c7f1a5-4d84-491c-aae1-a91fa310d141 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:27:04.699178 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1432/2853] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 14:27:04 2022] DELETE /identity/v3/projects/b78358f25fbf41569b73f7500baee3cc => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:27:14.343921 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=43,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:27:14.693278 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=59,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:27:22.631089 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad525ea4-d0cc-4e9b-8558-db48922b0b33 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:27:22.632520 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-ad525ea4-d0cc-4e9b-8558-db48922b0b33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:27:22.645628 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad525ea4-d0cc-4e9b-8558-db48922b0b33 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:27:22.702942 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b368776a-479b-401f-9523-cb1e6c7c4c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:27:22.703626 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b368776a-479b-401f-9523-cb1e6c7c4c02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:27:22.703908 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b368776a-479b-401f-9523-cb1e6c7c4c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:27:22.704181 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b368776a-479b-401f-9523-cb1e6c7c4c02 admin admin] PATH_INFO: `/v3/users/fa533a8124b64affa12b7d624c778da7` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:27:22.705382 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b368776a-479b-401f-9523-cb1e6c7c4c02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa533a8124b64affa12b7d624c778da7)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:27:22.707442 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b368776a-479b-401f-9523-cb1e6c7c4c02 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:27:22.760660 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-b368776a-479b-401f-9523-cb1e6c7c4c02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa533a8124b64affa12b7d624c778da7'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:27:22.761354 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-b368776a-479b-401f-9523-cb1e6c7c4c02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa533a8124b64affa12b7d624c778da7'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:27:22.768605 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-b368776a-479b-401f-9523-cb1e6c7c4c02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa533a8124b64affa12b7d624c778da7'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:27:22.769302 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-b368776a-479b-401f-9523-cb1e6c7c4c02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa533a8124b64affa12b7d624c778da7'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:27:22.782194 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-b368776a-479b-401f-9523-cb1e6c7c4c02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa533a8124b64affa12b7d624c778da7'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:27:22.797269 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1422/2854] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 14:27:22 2022] DELETE /identity/v3/users/fa533a8124b64affa12b7d624c778da7 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:27:22.802868 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cfb1ea-6db8-4082-b483-33b46679f0b1 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:27:22.803399 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cfb1ea-6db8-4082-b483-33b46679f0b1 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:27:22.803491 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cfb1ea-6db8-4082-b483-33b46679f0b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:27:22.823122 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-50cfb1ea-6db8-4082-b483-33b46679f0b1 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:27:22.824327 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-50cfb1ea-6db8-4082-b483-33b46679f0b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:27:22.887261 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1433/2855] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:27:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:27:22.896980 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b368776a-479b-401f-9523-cb1e6c7c4c02 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:27:22.898569 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-b368776a-479b-401f-9523-cb1e6c7c4c02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:27:22.959289 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b368776a-479b-401f-9523-cb1e6c7c4c02 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:27:22.965041 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca1e1ad-88b9-4861-aa74-fc7b3f89a325 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:27:22.965726 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca1e1ad-88b9-4861-aa74-fc7b3f89a325 service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:27:22.966013 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca1e1ad-88b9-4861-aa74-fc7b3f89a325 service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:27:22.966290 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca1e1ad-88b9-4861-aa74-fc7b3f89a325 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:27:22.976841 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca1e1ad-88b9-4861-aa74-fc7b3f89a325 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:27:22.978354 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca1e1ad-88b9-4861-aa74-fc7b3f89a325 service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:27:22.988687 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1423/2856] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:27:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:27:23.262971 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cfb1ea-6db8-4082-b483-33b46679f0b1 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:27:23.264910 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-50cfb1ea-6db8-4082-b483-33b46679f0b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:27:23.277990 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cfb1ea-6db8-4082-b483-33b46679f0b1 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:27:23.283544 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205016de-da46-4737-b817-25da22d2358c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:27:23.284276 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205016de-da46-4737-b817-25da22d2358c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:27:23.284659 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205016de-da46-4737-b817-25da22d2358c admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:27:23.284943 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205016de-da46-4737-b817-25da22d2358c admin admin] PATH_INFO: `/v3/projects/d4c0f9d5d84a4ed2aa6368c212c8f3e1` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:27:23.286124 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205016de-da46-4737-b817-25da22d2358c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4c0f9d5d84a4ed2aa6368c212c8f3e1)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:27:23.288028 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205016de-da46-4737-b817-25da22d2358c admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:27:23.339460 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-205016de-da46-4737-b817-25da22d2358c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4c0f9d5d84a4ed2aa6368c212c8f3e1', 'request_id': 'req-205016de-da46-4737-b817-25da22d2358c'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:27:23.347794 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-205016de-da46-4737-b817-25da22d2358c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4c0f9d5d84a4ed2aa6368c212c8f3e1', 'request_id': 'req-205016de-da46-4737-b817-25da22d2358c'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:27:23.353715 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-205016de-da46-4737-b817-25da22d2358c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4c0f9d5d84a4ed2aa6368c212c8f3e1', 'request_id': 'req-205016de-da46-4737-b817-25da22d2358c'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:27:23.365058 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-205016de-da46-4737-b817-25da22d2358c admin admin] The token cache is being invalidate because project d4c0f9d5d84a4ed2aa6368c212c8f3e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86933) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:27:23.365345 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-205016de-da46-4737-b817-25da22d2358c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:27:23.366009 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-205016de-da46-4737-b817-25da22d2358c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:27:23.367514 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1434/2857] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 14:27:23 2022] DELETE /identity/v3/projects/d4c0f9d5d84a4ed2aa6368c212c8f3e1 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:27:23.372374 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d387f33-c33c-452a-8a9f-6feb5793d15b None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:27:23.372667 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d387f33-c33c-452a-8a9f-6feb5793d15b None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:27:23.372945 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d387f33-c33c-452a-8a9f-6feb5793d15b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:27:23.393240 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-4d387f33-c33c-452a-8a9f-6feb5793d15b None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:27:23.394409 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-4d387f33-c33c-452a-8a9f-6feb5793d15b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:27:23.410948 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:27:23.455965 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1424/2858] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:27:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:27:23.465103 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedfa8cb-d830-4efe-911a-1b3287576b7e None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:27:23.465389 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedfa8cb-d830-4efe-911a-1b3287576b7e None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:27:23.465770 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedfa8cb-d830-4efe-911a-1b3287576b7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:27:23.484980 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-cedfa8cb-d830-4efe-911a-1b3287576b7e None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:27:23.486205 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-cedfa8cb-d830-4efe-911a-1b3287576b7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:27:23.493962 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1435/2859] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:27:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:27:23.498240 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d387f33-c33c-452a-8a9f-6feb5793d15b None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:27:23.508966 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d387f33-c33c-452a-8a9f-6feb5793d15b None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:27:23.519164 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2d39a1-b27c-4222-a1e3-0c7d82777474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:27:23.519787 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2d39a1-b27c-4222-a1e3-0c7d82777474 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:27:23.520071 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2d39a1-b27c-4222-a1e3-0c7d82777474 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:27:23.520347 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2d39a1-b27c-4222-a1e3-0c7d82777474 admin admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:27:23.521053 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2d39a1-b27c-4222-a1e3-0c7d82777474 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:27:23.521367 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2d39a1-b27c-4222-a1e3-0c7d82777474 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:27:23.523018 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2d39a1-b27c-4222-a1e3-0c7d82777474 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:27:23.535124 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1425/2860] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Jul 26 14:27:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:27:23.539591 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedfa8cb-d830-4efe-911a-1b3287576b7e None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:27:23.549046 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedfa8cb-d830-4efe-911a-1b3287576b7e None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:27:23.553647 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bd9a6a-92c4-4f89-b064-c6ed0c706c18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:27:23.554255 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bd9a6a-92c4-4f89-b064-c6ed0c706c18 admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:27:23.554539 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bd9a6a-92c4-4f89-b064-c6ed0c706c18 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:27:23.554810 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bd9a6a-92c4-4f89-b064-c6ed0c706c18 admin admin] PATH_INFO: `/v3/projects` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:27:23.555468 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bd9a6a-92c4-4f89-b064-c6ed0c706c18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:27:23.557351 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bd9a6a-92c4-4f89-b064-c6ed0c706c18 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:27:23.584774 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1436/2861] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 14:27:23 2022] POST /identity/v3/projects => generated 359 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:27:23.588901 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2d39a1-b27c-4222-a1e3-0c7d82777474 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:27:23.598830 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2d39a1-b27c-4222-a1e3-0c7d82777474 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:27:23.664367 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612781f3-067e-4f6f-94ad-d18021502ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:27:23.665161 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612781f3-067e-4f6f-94ad-d18021502ee6 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:27:23.665161 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612781f3-067e-4f6f-94ad-d18021502ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:27:23.665396 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612781f3-067e-4f6f-94ad-d18021502ee6 admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:27:23.666113 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612781f3-067e-4f6f-94ad-d18021502ee6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:27:23.668082 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612781f3-067e-4f6f-94ad-d18021502ee6 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:27:23.692191 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1426/2862] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 14:27:23 2022] POST /identity/v3/users => generated 280 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:27:23.696440 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bd9a6a-92c4-4f89-b064-c6ed0c706c18 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:27:23.707920 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bd9a6a-92c4-4f89-b064-c6ed0c706c18 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:27:23.712706 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a68219-565e-42d7-9ac3-9e848da58d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:27:23.713328 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a68219-565e-42d7-9ac3-9e848da58d2f admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:27:23.713662 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a68219-565e-42d7-9ac3-9e848da58d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:27:23.713942 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a68219-565e-42d7-9ac3-9e848da58d2f admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:27:23.714641 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a68219-565e-42d7-9ac3-9e848da58d2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:27:23.716227 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a68219-565e-42d7-9ac3-9e848da58d2f admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:27:23.726520 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1437/2863] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:27:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:27:23.730759 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612781f3-067e-4f6f-94ad-d18021502ee6 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:27:23.745068 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612781f3-067e-4f6f-94ad-d18021502ee6 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:27:23.750030 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063ba1c0-8590-4dd8-b2b9-6595715b3871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:27:23.751152 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063ba1c0-8590-4dd8-b2b9-6595715b3871 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:27:23.751434 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063ba1c0-8590-4dd8-b2b9-6595715b3871 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:27:23.751707 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063ba1c0-8590-4dd8-b2b9-6595715b3871 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:27:23.752403 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063ba1c0-8590-4dd8-b2b9-6595715b3871 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:27:23.754147 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063ba1c0-8590-4dd8-b2b9-6595715b3871 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:27:23.765874 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1427/2864] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:27:23 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 14:27:23.770664 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a68219-565e-42d7-9ac3-9e848da58d2f admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:27:23.781721 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a68219-565e-42d7-9ac3-9e848da58d2f admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:27:23.786688 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae853a8-3b9d-45b7-8550-01053d3dd05f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:27:23.787368 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae853a8-3b9d-45b7-8550-01053d3dd05f admin admin] REQUEST_METHOD: `PUT` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:27:23.787654 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae853a8-3b9d-45b7-8550-01053d3dd05f admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:27:23.787932 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae853a8-3b9d-45b7-8550-01053d3dd05f admin admin] PATH_INFO: `/v3/projects/8c5560b8961f496d8261ffa8011046a2/users/f0c095e081c847b8b6f63518cb78cb87/roles/759f389b221049c4b01faf3b984cd8d7` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:27:23.803669 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae853a8-3b9d-45b7-8550-01053d3dd05f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c5560b8961f496d8261ffa8011046a2, user_id=f0c095e081c847b8b6f63518cb78cb87, role_id=759f389b221049c4b01faf3b984cd8d7)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:27:23.806364 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae853a8-3b9d-45b7-8550-01053d3dd05f admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:27:23.821770 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1438/2865] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 14:27:23 2022] PUT /identity/v3/projects/8c5560b8961f496d8261ffa8011046a2/users/f0c095e081c847b8b6f63518cb78cb87/roles/759f389b221049c4b01faf3b984cd8d7 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:27:23.832600 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3cab31-494a-4d3f-80de-3651b3c6e9ef None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:27:23.832886 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3cab31-494a-4d3f-80de-3651b3c6e9ef None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:27:23.833165 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3cab31-494a-4d3f-80de-3651b3c6e9ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:27:23.863172 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-0e3cab31-494a-4d3f-80de-3651b3c6e9ef None None] MFA Rules not processed for user `f0c095e081c847b8b6f63518cb78cb87`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:27:23.864439 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-0e3cab31-494a-4d3f-80de-3651b3c6e9ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:27:23.931116 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1428/2866] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:27:23 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:27:23.964087 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae853a8-3b9d-45b7-8550-01053d3dd05f admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:27:23.966150 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-bae853a8-3b9d-45b7-8550-01053d3dd05f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:27:23.988323 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:27:24.025205 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae853a8-3b9d-45b7-8550-01053d3dd05f admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:27:24.028882 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3323b505-9cc2-4d4e-8fa9-3fac2b9b9e35 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:27:24.029524 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3323b505-9cc2-4d4e-8fa9-3fac2b9b9e35 service cinder] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:27:24.029816 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3323b505-9cc2-4d4e-8fa9-3fac2b9b9e35 service cinder] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:27:24.030087 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3323b505-9cc2-4d4e-8fa9-3fac2b9b9e35 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:27:24.039019 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3323b505-9cc2-4d4e-8fa9-3fac2b9b9e35 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:27:24.040419 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3323b505-9cc2-4d4e-8fa9-3fac2b9b9e35 service cinder] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:27:24.050069 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1439/2867] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:27:23 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:27:24.065574 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3cab31-494a-4d3f-80de-3651b3c6e9ef None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:27:24.084358 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:27:24.129512 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3cab31-494a-4d3f-80de-3651b3c6e9ef None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:27:24.134320 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa35280-82bd-4446-82c4-9088a2865668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:27:24.134959 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa35280-82bd-4446-82c4-9088a2865668 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:27:24.135246 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa35280-82bd-4446-82c4-9088a2865668 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:27:24.135523 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa35280-82bd-4446-82c4-9088a2865668 admin admin] PATH_INFO: `/v3/users/f0c095e081c847b8b6f63518cb78cb87` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:27:24.136686 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa35280-82bd-4446-82c4-9088a2865668 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0c095e081c847b8b6f63518cb78cb87)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:27:24.138638 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa35280-82bd-4446-82c4-9088a2865668 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:27:24.185686 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-1aa35280-82bd-4446-82c4-9088a2865668 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0c095e081c847b8b6f63518cb78cb87'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:27:24.186278 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-1aa35280-82bd-4446-82c4-9088a2865668 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0c095e081c847b8b6f63518cb78cb87'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:27:24.193223 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-1aa35280-82bd-4446-82c4-9088a2865668 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0c095e081c847b8b6f63518cb78cb87'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:27:24.193804 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-1aa35280-82bd-4446-82c4-9088a2865668 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0c095e081c847b8b6f63518cb78cb87'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:27:24.206200 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-1aa35280-82bd-4446-82c4-9088a2865668 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0c095e081c847b8b6f63518cb78cb87'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:27:24.220696 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1429/2868] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 14:27:24 2022] DELETE /identity/v3/users/f0c095e081c847b8b6f63518cb78cb87 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:27:24.225298 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f669d1-35d7-47e4-8595-08d0f3a1e4e1 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:27:24.225722 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f669d1-35d7-47e4-8595-08d0f3a1e4e1 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:27:24.226098 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f669d1-35d7-47e4-8595-08d0f3a1e4e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:27:24.246136 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-b8f669d1-35d7-47e4-8595-08d0f3a1e4e1 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:27:24.247423 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-b8f669d1-35d7-47e4-8595-08d0f3a1e4e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:27:24.310918 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1440/2869] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:27:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:27:24.320358 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa35280-82bd-4446-82c4-9088a2865668 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:27:24.321937 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-1aa35280-82bd-4446-82c4-9088a2865668 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:27:24.380149 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa35280-82bd-4446-82c4-9088a2865668 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:27:24.384114 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b671eb01-1b6b-4a99-b7da-38a974ebb9a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:27:24.384730 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b671eb01-1b6b-4a99-b7da-38a974ebb9a8 service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:27:24.385012 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b671eb01-1b6b-4a99-b7da-38a974ebb9a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:27:24.385289 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b671eb01-1b6b-4a99-b7da-38a974ebb9a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:27:24.394405 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b671eb01-1b6b-4a99-b7da-38a974ebb9a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:27:24.395837 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b671eb01-1b6b-4a99-b7da-38a974ebb9a8 service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:27:24.405459 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1430/2870] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:27:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 14:27:24.674784 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f669d1-35d7-47e4-8595-08d0f3a1e4e1 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:27:24.676226 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-b8f669d1-35d7-47e4-8595-08d0f3a1e4e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:27:24.694531 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f669d1-35d7-47e4-8595-08d0f3a1e4e1 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:27:24.699980 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687f21d4-e64f-4161-af77-07f7c0e191bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:27:24.700670 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687f21d4-e64f-4161-af77-07f7c0e191bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:27:24.700917 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687f21d4-e64f-4161-af77-07f7c0e191bc admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:27:24.701196 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687f21d4-e64f-4161-af77-07f7c0e191bc admin admin] PATH_INFO: `/v3/projects/8c5560b8961f496d8261ffa8011046a2` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:27:24.702549 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687f21d4-e64f-4161-af77-07f7c0e191bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c5560b8961f496d8261ffa8011046a2)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:27:24.705042 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687f21d4-e64f-4161-af77-07f7c0e191bc admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:27:24.751502 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-687f21d4-e64f-4161-af77-07f7c0e191bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c5560b8961f496d8261ffa8011046a2', 'request_id': 'req-687f21d4-e64f-4161-af77-07f7c0e191bc'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:27:24.759839 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-687f21d4-e64f-4161-af77-07f7c0e191bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c5560b8961f496d8261ffa8011046a2', 'request_id': 'req-687f21d4-e64f-4161-af77-07f7c0e191bc'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:27:24.765803 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-687f21d4-e64f-4161-af77-07f7c0e191bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c5560b8961f496d8261ffa8011046a2', 'request_id': 'req-687f21d4-e64f-4161-af77-07f7c0e191bc'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:27:24.777527 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-687f21d4-e64f-4161-af77-07f7c0e191bc admin admin] The token cache is being invalidate because project 8c5560b8961f496d8261ffa8011046a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86933) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:27:24.777820 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-687f21d4-e64f-4161-af77-07f7c0e191bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:27:24.778473 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-687f21d4-e64f-4161-af77-07f7c0e191bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:27:24.779990 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1441/2871] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 14:27:24 2022] DELETE /identity/v3/projects/8c5560b8961f496d8261ffa8011046a2 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:27:24.785645 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4458c32-8e69-4518-abc2-88877a2ad85f None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:27:24.786105 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4458c32-8e69-4518-abc2-88877a2ad85f None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:27:24.786210 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4458c32-8e69-4518-abc2-88877a2ad85f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:27:24.805356 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-d4458c32-8e69-4518-abc2-88877a2ad85f None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:27:24.806613 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-d4458c32-8e69-4518-abc2-88877a2ad85f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:27:24.842570 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:27:24.872985 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1431/2872] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:27:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:27:24.881758 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1188dcec-16dc-416d-84dc-353cea570a22 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:27:24.882251 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1188dcec-16dc-416d-84dc-353cea570a22 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:27:24.882359 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1188dcec-16dc-416d-84dc-353cea570a22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:27:24.903778 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-1188dcec-16dc-416d-84dc-353cea570a22 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:27:24.904995 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-1188dcec-16dc-416d-84dc-353cea570a22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:27:24.917443 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1442/2873] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:27:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:27:24.917889 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4458c32-8e69-4518-abc2-88877a2ad85f None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:27:24.929363 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4458c32-8e69-4518-abc2-88877a2ad85f None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:27:24.936049 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dc2470-f043-4dbb-adc4-e92996163ab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:27:24.936675 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dc2470-f043-4dbb-adc4-e92996163ab7 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:27:24.936957 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dc2470-f043-4dbb-adc4-e92996163ab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:27:24.937229 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dc2470-f043-4dbb-adc4-e92996163ab7 admin admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:27:24.938495 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dc2470-f043-4dbb-adc4-e92996163ab7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:27:24.938806 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dc2470-f043-4dbb-adc4-e92996163ab7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:27:24.940399 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dc2470-f043-4dbb-adc4-e92996163ab7 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:27:24.960645 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1432/2874] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Jul 26 14:27:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:27:24.965120 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1188dcec-16dc-416d-84dc-353cea570a22 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:27:24.974644 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1188dcec-16dc-416d-84dc-353cea570a22 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:27:24.979763 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422beae9-6e02-484d-9bd6-53e30b45cb6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:27:24.980405 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422beae9-6e02-484d-9bd6-53e30b45cb6e admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:27:24.980683 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422beae9-6e02-484d-9bd6-53e30b45cb6e admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:27:24.980951 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422beae9-6e02-484d-9bd6-53e30b45cb6e admin admin] PATH_INFO: `/v3/projects` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:27:24.981658 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422beae9-6e02-484d-9bd6-53e30b45cb6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:27:24.983518 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422beae9-6e02-484d-9bd6-53e30b45cb6e admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:27:25.018445 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1443/2875] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 14:27:24 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 14:27:25.023137 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dc2470-f043-4dbb-adc4-e92996163ab7 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:27:25.033564 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dc2470-f043-4dbb-adc4-e92996163ab7 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:27:25.094005 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e147307-e444-46b0-b569-d2ec4cd59105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:27:25.094630 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e147307-e444-46b0-b569-d2ec4cd59105 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:27:25.094914 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e147307-e444-46b0-b569-d2ec4cd59105 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:27:25.095190 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e147307-e444-46b0-b569-d2ec4cd59105 admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:27:25.095870 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e147307-e444-46b0-b569-d2ec4cd59105 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:27:25.097810 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e147307-e444-46b0-b569-d2ec4cd59105 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:27:25.126055 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1433/2876] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 14:27:25 2022] POST /identity/v3/users => generated 292 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:27:25.130192 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422beae9-6e02-484d-9bd6-53e30b45cb6e admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:27:25.146241 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422beae9-6e02-484d-9bd6-53e30b45cb6e admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:27:25.151284 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91e38d9-e7ed-4a2b-b3cf-3ae9c55ca87f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:27:25.151996 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91e38d9-e7ed-4a2b-b3cf-3ae9c55ca87f admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:27:25.152499 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91e38d9-e7ed-4a2b-b3cf-3ae9c55ca87f admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:27:25.152901 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91e38d9-e7ed-4a2b-b3cf-3ae9c55ca87f admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:27:25.153712 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91e38d9-e7ed-4a2b-b3cf-3ae9c55ca87f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:27:25.155370 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91e38d9-e7ed-4a2b-b3cf-3ae9c55ca87f admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:27:25.169954 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1444/2877] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:27:25 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:27:25.172341 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e147307-e444-46b0-b569-d2ec4cd59105 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:27:25.183623 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e147307-e444-46b0-b569-d2ec4cd59105 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:27:25.189888 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd430570-fc69-40d5-82f4-b9d434007baf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:27:25.190784 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd430570-fc69-40d5-82f4-b9d434007baf admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:27:25.191579 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd430570-fc69-40d5-82f4-b9d434007baf admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:27:25.192064 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd430570-fc69-40d5-82f4-b9d434007baf admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:27:25.192924 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd430570-fc69-40d5-82f4-b9d434007baf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:27:25.194803 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd430570-fc69-40d5-82f4-b9d434007baf admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:27:25.209629 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1434/2878] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:27:25 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:27:25.213905 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91e38d9-e7ed-4a2b-b3cf-3ae9c55ca87f admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:27:25.225319 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91e38d9-e7ed-4a2b-b3cf-3ae9c55ca87f admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:27:25.230091 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0330a11-3199-4f59-8c2e-1ce3c856cec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:27:25.230979 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0330a11-3199-4f59-8c2e-1ce3c856cec2 admin admin] REQUEST_METHOD: `PUT` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:27:25.231340 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0330a11-3199-4f59-8c2e-1ce3c856cec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:27:25.231621 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0330a11-3199-4f59-8c2e-1ce3c856cec2 admin admin] PATH_INFO: `/v3/projects/f28e2708708a49aeb074bd7f4e81ae28/users/0a5ab342bcaa40e1a4c6308efbbab699/roles/759f389b221049c4b01faf3b984cd8d7` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:27:25.234128 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:27:25.251341 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0330a11-3199-4f59-8c2e-1ce3c856cec2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f28e2708708a49aeb074bd7f4e81ae28, user_id=0a5ab342bcaa40e1a4c6308efbbab699, role_id=759f389b221049c4b01faf3b984cd8d7)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:27:25.254029 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0330a11-3199-4f59-8c2e-1ce3c856cec2 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:27:25.268852 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1445/2879] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 14:27:25 2022] PUT /identity/v3/projects/f28e2708708a49aeb074bd7f4e81ae28/users/0a5ab342bcaa40e1a4c6308efbbab699/roles/759f389b221049c4b01faf3b984cd8d7 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:27:25.280235 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56246a4f-31d5-412b-ba99-ed31dd987443 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:27:25.280607 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56246a4f-31d5-412b-ba99-ed31dd987443 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:27:25.280887 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56246a4f-31d5-412b-ba99-ed31dd987443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:27:25.314405 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-56246a4f-31d5-412b-ba99-ed31dd987443 None None] MFA Rules not processed for user `0a5ab342bcaa40e1a4c6308efbbab699`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:27:25.315780 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-56246a4f-31d5-412b-ba99-ed31dd987443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:27:25.386118 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1435/2880] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:27:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:27:25.390631 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0330a11-3199-4f59-8c2e-1ce3c856cec2 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:27:25.451576 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0330a11-3199-4f59-8c2e-1ce3c856cec2 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:27:25.456217 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b16be8-0afc-498d-b6da-02f622beebef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:27:25.456841 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b16be8-0afc-498d-b6da-02f622beebef admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:27:25.457126 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b16be8-0afc-498d-b6da-02f622beebef admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:27:25.457404 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b16be8-0afc-498d-b6da-02f622beebef admin admin] PATH_INFO: `/v3/projects` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:27:25.458096 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b16be8-0afc-498d-b6da-02f622beebef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:27:25.459981 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b16be8-0afc-498d-b6da-02f622beebef admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:27:25.487610 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1446/2881] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 14:27:25 2022] POST /identity/v3/projects => generated 383 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:27:25.491793 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56246a4f-31d5-412b-ba99-ed31dd987443 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:27:25.503193 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56246a4f-31d5-412b-ba99-ed31dd987443 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:27:25.535922 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:27:25.561319 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ece30e-62a3-4e7d-883f-88c3f0784ba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:27:25.561974 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ece30e-62a3-4e7d-883f-88c3f0784ba0 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:27:25.562258 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ece30e-62a3-4e7d-883f-88c3f0784ba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:27:25.562534 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ece30e-62a3-4e7d-883f-88c3f0784ba0 admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:27:25.563910 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ece30e-62a3-4e7d-883f-88c3f0784ba0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:27:25.565849 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ece30e-62a3-4e7d-883f-88c3f0784ba0 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:27:25.587084 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1436/2882] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 14:27:25 2022] POST /identity/v3/users => generated 292 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:27:25.591431 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b16be8-0afc-498d-b6da-02f622beebef admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:27:25.601820 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b16be8-0afc-498d-b6da-02f622beebef admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:27:25.606393 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471232a4-22ce-4d08-b8c6-fb642799af11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:27:25.607003 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471232a4-22ce-4d08-b8c6-fb642799af11 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:27:25.607288 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471232a4-22ce-4d08-b8c6-fb642799af11 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:27:25.607568 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471232a4-22ce-4d08-b8c6-fb642799af11 admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:27:25.608258 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471232a4-22ce-4d08-b8c6-fb642799af11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:27:25.610467 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471232a4-22ce-4d08-b8c6-fb642799af11 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:27:25.620910 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1447/2883] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:27:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 14:27:25.624765 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ece30e-62a3-4e7d-883f-88c3f0784ba0 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:27:25.635746 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ece30e-62a3-4e7d-883f-88c3f0784ba0 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:27:25.640467 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56309a25-c05f-40e9-be82-9dc67b12d770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:27:25.641124 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56309a25-c05f-40e9-be82-9dc67b12d770 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:27:25.641416 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56309a25-c05f-40e9-be82-9dc67b12d770 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:27:25.641722 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56309a25-c05f-40e9-be82-9dc67b12d770 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:27:25.642414 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56309a25-c05f-40e9-be82-9dc67b12d770 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:27:25.643988 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56309a25-c05f-40e9-be82-9dc67b12d770 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:27:25.654039 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1437/2884] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:27:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:27:25.658494 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471232a4-22ce-4d08-b8c6-fb642799af11 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:27:25.668399 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471232a4-22ce-4d08-b8c6-fb642799af11 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:27:25.673728 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa18bb0-c597-4fa8-a749-0635359ac025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:27:25.674514 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa18bb0-c597-4fa8-a749-0635359ac025 admin admin] REQUEST_METHOD: `PUT` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:27:25.674794 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa18bb0-c597-4fa8-a749-0635359ac025 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:27:25.675070 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa18bb0-c597-4fa8-a749-0635359ac025 admin admin] PATH_INFO: `/v3/projects/2a2ff5bb83d24a39939cb063159b2b68/users/ed2d7bfba97c4dd7a5280e4cc0b129a8/roles/759f389b221049c4b01faf3b984cd8d7` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:27:25.689699 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa18bb0-c597-4fa8-a749-0635359ac025 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a2ff5bb83d24a39939cb063159b2b68, user_id=ed2d7bfba97c4dd7a5280e4cc0b129a8, role_id=759f389b221049c4b01faf3b984cd8d7)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:27:25.692328 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa18bb0-c597-4fa8-a749-0635359ac025 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:27:25.705183 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1448/2885] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 14:27:25 2022] PUT /identity/v3/projects/2a2ff5bb83d24a39939cb063159b2b68/users/ed2d7bfba97c4dd7a5280e4cc0b129a8/roles/759f389b221049c4b01faf3b984cd8d7 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:27:25.715911 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eb4a24-2832-48a5-8fd6-5b1656fadb9d None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:27:25.716412 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eb4a24-2832-48a5-8fd6-5b1656fadb9d None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:27:25.716798 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eb4a24-2832-48a5-8fd6-5b1656fadb9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:27:25.748608 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-a8eb4a24-2832-48a5-8fd6-5b1656fadb9d None None] MFA Rules not processed for user `ed2d7bfba97c4dd7a5280e4cc0b129a8`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:27:25.749851 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-a8eb4a24-2832-48a5-8fd6-5b1656fadb9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:27:25.813792 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1438/2886] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:27:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:27:25.818385 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa18bb0-c597-4fa8-a749-0635359ac025 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:27:25.879268 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa18bb0-c597-4fa8-a749-0635359ac025 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:27:25.884177 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15b2dec-9b2e-4072-9621-43ed6fea516e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:27:25.884812 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b2dec-9b2e-4072-9621-43ed6fea516e admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:27:25.885074 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b2dec-9b2e-4072-9621-43ed6fea516e admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:27:25.885353 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b2dec-9b2e-4072-9621-43ed6fea516e admin admin] PATH_INFO: `/v3/projects` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:27:25.886133 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15b2dec-9b2e-4072-9621-43ed6fea516e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:27:25.888004 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15b2dec-9b2e-4072-9621-43ed6fea516e admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:27:25.916128 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1449/2887] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 14:27:25 2022] POST /identity/v3/projects => generated 383 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:27:25.920818 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eb4a24-2832-48a5-8fd6-5b1656fadb9d None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:27:25.932297 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eb4a24-2832-48a5-8fd6-5b1656fadb9d None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:27:25.987980 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c99bb8-4798-4ff3-babc-b6b72ec4f6ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:27:25.988599 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c99bb8-4798-4ff3-babc-b6b72ec4f6ea admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:27:25.988884 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c99bb8-4798-4ff3-babc-b6b72ec4f6ea admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:27:25.989163 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c99bb8-4798-4ff3-babc-b6b72ec4f6ea admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:27:25.989871 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c99bb8-4798-4ff3-babc-b6b72ec4f6ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:27:25.991789 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c99bb8-4798-4ff3-babc-b6b72ec4f6ea admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:27:26.012814 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1439/2888] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 14:27:25 2022] POST /identity/v3/users => generated 298 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:27:26.017355 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15b2dec-9b2e-4072-9621-43ed6fea516e admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:27:26.027852 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15b2dec-9b2e-4072-9621-43ed6fea516e admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:27:26.033412 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97944181-1fb0-436f-af40-eac6280d83d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:27:26.034061 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97944181-1fb0-436f-af40-eac6280d83d8 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:27:26.034346 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97944181-1fb0-436f-af40-eac6280d83d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:27:26.034624 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97944181-1fb0-436f-af40-eac6280d83d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:27:26.035322 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97944181-1fb0-436f-af40-eac6280d83d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:27:26.036921 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97944181-1fb0-436f-af40-eac6280d83d8 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:27:26.047799 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1450/2889] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:27:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:27:26.051991 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c99bb8-4798-4ff3-babc-b6b72ec4f6ea admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:27:26.062741 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c99bb8-4798-4ff3-babc-b6b72ec4f6ea admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:27:26.067439 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6599b86-ce6d-4999-8f85-5e58205e5956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:27:26.068054 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6599b86-ce6d-4999-8f85-5e58205e5956 admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:27:26.068341 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6599b86-ce6d-4999-8f85-5e58205e5956 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:27:26.068622 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6599b86-ce6d-4999-8f85-5e58205e5956 admin admin] PATH_INFO: `/v3/projects/88535297250b4ec698234db72ab168b9/users/a0893c93f4b0444bb11c5d13b28fcca9/roles/6e0201a9a6484f7d8b112b1eafff70bd` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:27:26.083866 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6599b86-ce6d-4999-8f85-5e58205e5956 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88535297250b4ec698234db72ab168b9, user_id=a0893c93f4b0444bb11c5d13b28fcca9, role_id=6e0201a9a6484f7d8b112b1eafff70bd)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:27:26.086678 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6599b86-ce6d-4999-8f85-5e58205e5956 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:27:26.100561 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1440/2890] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 14:27:26 2022] PUT /identity/v3/projects/88535297250b4ec698234db72ab168b9/users/a0893c93f4b0444bb11c5d13b28fcca9/roles/6e0201a9a6484f7d8b112b1eafff70bd => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:27:26.104518 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97944181-1fb0-436f-af40-eac6280d83d8 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:27:26.114337 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97944181-1fb0-436f-af40-eac6280d83d8 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:27:26.117841 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:27:26.170211 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f15084-316d-425d-82d4-1e61cb56c74a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:27:26.170919 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f15084-316d-425d-82d4-1e61cb56c74a admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:27:26.171116 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f15084-316d-425d-82d4-1e61cb56c74a admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:27:26.171436 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f15084-316d-425d-82d4-1e61cb56c74a admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:27:26.172156 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f15084-316d-425d-82d4-1e61cb56c74a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:27:26.173756 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f15084-316d-425d-82d4-1e61cb56c74a admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:27:26.183149 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1451/2891] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:27:26 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:27:26.187367 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6599b86-ce6d-4999-8f85-5e58205e5956 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:27:26.197633 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6599b86-ce6d-4999-8f85-5e58205e5956 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:27:26.202250 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b779768b-2037-4dae-80d7-42a8018258e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:27:26.202908 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b779768b-2037-4dae-80d7-42a8018258e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:27:26.203186 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b779768b-2037-4dae-80d7-42a8018258e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:27:26.203952 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b779768b-2037-4dae-80d7-42a8018258e2 admin admin] PATH_INFO: `/v3/projects/88535297250b4ec698234db72ab168b9/users/a0893c93f4b0444bb11c5d13b28fcca9/roles/6e0201a9a6484f7d8b112b1eafff70bd` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:27:26.206050 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b779768b-2037-4dae-80d7-42a8018258e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88535297250b4ec698234db72ab168b9, user_id=a0893c93f4b0444bb11c5d13b28fcca9, role_id=6e0201a9a6484f7d8b112b1eafff70bd)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:27:26.208768 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b779768b-2037-4dae-80d7-42a8018258e2 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:27:26.212623 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:27:26.222253 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1441/2892] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 14:27:26 2022] PUT /identity/v3/projects/88535297250b4ec698234db72ab168b9/users/a0893c93f4b0444bb11c5d13b28fcca9/roles/6e0201a9a6484f7d8b112b1eafff70bd => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:27:26.233654 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a69f11f-cb31-4e35-a44d-6f9cca1c8f38 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:27:26.233942 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a69f11f-cb31-4e35-a44d-6f9cca1c8f38 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:27:26.234221 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a69f11f-cb31-4e35-a44d-6f9cca1c8f38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:27:26.261572 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-2a69f11f-cb31-4e35-a44d-6f9cca1c8f38 None None] MFA Rules not processed for user `a0893c93f4b0444bb11c5d13b28fcca9`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:27:26.262780 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-2a69f11f-cb31-4e35-a44d-6f9cca1c8f38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:27:26.338996 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1452/2893] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:27:26 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:27:26.351109 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b779768b-2037-4dae-80d7-42a8018258e2 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:27:26.352868 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-b779768b-2037-4dae-80d7-42a8018258e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:27:26.412012 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b779768b-2037-4dae-80d7-42a8018258e2 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:27:26.415983 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1254390-3719-4ba0-ac6f-dd98b68e4333 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:27:26.416596 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1254390-3719-4ba0-ac6f-dd98b68e4333 service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:27:26.416875 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1254390-3719-4ba0-ac6f-dd98b68e4333 service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:27:26.417146 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1254390-3719-4ba0-ac6f-dd98b68e4333 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:27:26.472925 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1254390-3719-4ba0-ac6f-dd98b68e4333 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:27:26.474382 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1254390-3719-4ba0-ac6f-dd98b68e4333 service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:27:26.483102 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1442/2894] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:27:26 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:27:27.989073 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a69f11f-cb31-4e35-a44d-6f9cca1c8f38 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:27:27.990510 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-2a69f11f-cb31-4e35-a44d-6f9cca1c8f38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:27:28.003540 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a69f11f-cb31-4e35-a44d-6f9cca1c8f38 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:27:28.007526 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a858bd-59f4-4de0-bcca-96dbbbdd9125 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:27:28.008224 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a858bd-59f4-4de0-bcca-96dbbbdd9125 service cinder] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:27:28.008503 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a858bd-59f4-4de0-bcca-96dbbbdd9125 service cinder] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:27:28.008777 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a858bd-59f4-4de0-bcca-96dbbbdd9125 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:27:28.067121 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a858bd-59f4-4de0-bcca-96dbbbdd9125 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:27:28.068564 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a858bd-59f4-4de0-bcca-96dbbbdd9125 service cinder] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:27:28.077490 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1453/2895] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:27:27 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 14:27:28.597872 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1254390-3719-4ba0-ac6f-dd98b68e4333 service cinder] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:27:28.606385 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1254390-3719-4ba0-ac6f-dd98b68e4333 service cinder] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:27:28.610407 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8ace84-1236-45e6-8bc6-ddbca2f9e8fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:27:28.611356 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8ace84-1236-45e6-8bc6-ddbca2f9e8fe service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:27:28.611645 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8ace84-1236-45e6-8bc6-ddbca2f9e8fe service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:27:28.611917 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8ace84-1236-45e6-8bc6-ddbca2f9e8fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:27:28.620856 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8ace84-1236-45e6-8bc6-ddbca2f9e8fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:27:28.622285 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8ace84-1236-45e6-8bc6-ddbca2f9e8fe service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:27:28.632497 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1443/2896] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:27:28 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:27:32.500574 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a858bd-59f4-4de0-bcca-96dbbbdd9125 service cinder] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:27:32.533088 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=100 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:27:32.566349 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a858bd-59f4-4de0-bcca-96dbbbdd9125 service cinder] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:27:32.570837 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c87641b-a438-4132-b3a4-14100e4b21d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:27:32.571482 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c87641b-a438-4132-b3a4-14100e4b21d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:27:32.571764 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c87641b-a438-4132-b3a4-14100e4b21d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:27:32.572040 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c87641b-a438-4132-b3a4-14100e4b21d0 admin admin] PATH_INFO: `/v3/users/0a5ab342bcaa40e1a4c6308efbbab699` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:27:32.573200 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c87641b-a438-4132-b3a4-14100e4b21d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a5ab342bcaa40e1a4c6308efbbab699)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:27:32.575199 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c87641b-a438-4132-b3a4-14100e4b21d0 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:27:32.623396 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-3c87641b-a438-4132-b3a4-14100e4b21d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a5ab342bcaa40e1a4c6308efbbab699'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:27:32.632090 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-3c87641b-a438-4132-b3a4-14100e4b21d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a5ab342bcaa40e1a4c6308efbbab699'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:27:32.639796 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-3c87641b-a438-4132-b3a4-14100e4b21d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a5ab342bcaa40e1a4c6308efbbab699'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:27:32.640598 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-3c87641b-a438-4132-b3a4-14100e4b21d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a5ab342bcaa40e1a4c6308efbbab699'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:27:32.642005 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-3c87641b-a438-4132-b3a4-14100e4b21d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a5ab342bcaa40e1a4c6308efbbab699'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:27:32.660526 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1454/2897] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 14:27:32 2022] DELETE /identity/v3/users/0a5ab342bcaa40e1a4c6308efbbab699 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:27:32.665994 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdebd03-a742-4f70-b113-02576410c0f3 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:27:32.666489 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdebd03-a742-4f70-b113-02576410c0f3 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:27:32.666880 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdebd03-a742-4f70-b113-02576410c0f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:27:32.693216 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-2cdebd03-a742-4f70-b113-02576410c0f3 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:27:32.694490 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-2cdebd03-a742-4f70-b113-02576410c0f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:27:32.758667 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1444/2898] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:27:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:27:32.769087 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c87641b-a438-4132-b3a4-14100e4b21d0 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:27:32.770557 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-3c87641b-a438-4132-b3a4-14100e4b21d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:27:32.827623 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c87641b-a438-4132-b3a4-14100e4b21d0 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:27:32.831829 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546036aa-54d4-41aa-9c24-b1f0be27b8a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:27:32.832448 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546036aa-54d4-41aa-9c24-b1f0be27b8a3 service neutron] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:27:32.832729 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546036aa-54d4-41aa-9c24-b1f0be27b8a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:27:32.833005 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546036aa-54d4-41aa-9c24-b1f0be27b8a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:27:32.842036 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546036aa-54d4-41aa-9c24-b1f0be27b8a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:27:32.843460 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546036aa-54d4-41aa-9c24-b1f0be27b8a3 service neutron] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:27:32.853450 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1455/2899] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:27:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:27:33.130790 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdebd03-a742-4f70-b113-02576410c0f3 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:27:33.132198 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-2cdebd03-a742-4f70-b113-02576410c0f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:27:33.145910 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdebd03-a742-4f70-b113-02576410c0f3 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:27:33.150484 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93e5f90-9c5f-43c2-9b27-1171dc446f4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:27:33.151101 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93e5f90-9c5f-43c2-9b27-1171dc446f4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:27:33.151383 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93e5f90-9c5f-43c2-9b27-1171dc446f4c admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:27:33.151655 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93e5f90-9c5f-43c2-9b27-1171dc446f4c admin admin] PATH_INFO: `/v3/projects/f28e2708708a49aeb074bd7f4e81ae28` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:27:33.152803 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93e5f90-9c5f-43c2-9b27-1171dc446f4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f28e2708708a49aeb074bd7f4e81ae28)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:27:33.154722 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93e5f90-9c5f-43c2-9b27-1171dc446f4c admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:27:33.194193 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:27:33.207613 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-e93e5f90-9c5f-43c2-9b27-1171dc446f4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f28e2708708a49aeb074bd7f4e81ae28', 'request_id': 'req-e93e5f90-9c5f-43c2-9b27-1171dc446f4c'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:27:33.213933 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-e93e5f90-9c5f-43c2-9b27-1171dc446f4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f28e2708708a49aeb074bd7f4e81ae28', 'request_id': 'req-e93e5f90-9c5f-43c2-9b27-1171dc446f4c'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:27:33.219433 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-e93e5f90-9c5f-43c2-9b27-1171dc446f4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f28e2708708a49aeb074bd7f4e81ae28', 'request_id': 'req-e93e5f90-9c5f-43c2-9b27-1171dc446f4c'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:27:33.232139 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-e93e5f90-9c5f-43c2-9b27-1171dc446f4c admin admin] The token cache is being invalidate because project f28e2708708a49aeb074bd7f4e81ae28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:27:33.232423 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-e93e5f90-9c5f-43c2-9b27-1171dc446f4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:27:33.233066 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-e93e5f90-9c5f-43c2-9b27-1171dc446f4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:27:33.234592 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1445/2900] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 14:27:33 2022] DELETE /identity/v3/projects/f28e2708708a49aeb074bd7f4e81ae28 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:27:33.238910 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546036aa-54d4-41aa-9c24-b1f0be27b8a3 service neutron] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:27:33.248888 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546036aa-54d4-41aa-9c24-b1f0be27b8a3 service neutron] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:27:33.295774 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:27:33.309840 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8e9353-efd2-4294-b50a-21f1246e2e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:27:33.310494 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8e9353-efd2-4294-b50a-21f1246e2e4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:27:33.310778 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8e9353-efd2-4294-b50a-21f1246e2e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:27:33.311058 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8e9353-efd2-4294-b50a-21f1246e2e4e admin admin] PATH_INFO: `/v3/users/ed2d7bfba97c4dd7a5280e4cc0b129a8` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:27:33.312224 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8e9353-efd2-4294-b50a-21f1246e2e4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed2d7bfba97c4dd7a5280e4cc0b129a8)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:27:33.315192 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8e9353-efd2-4294-b50a-21f1246e2e4e admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:27:33.364606 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-9c8e9353-efd2-4294-b50a-21f1246e2e4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed2d7bfba97c4dd7a5280e4cc0b129a8'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:27:33.371540 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-9c8e9353-efd2-4294-b50a-21f1246e2e4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed2d7bfba97c4dd7a5280e4cc0b129a8'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:27:33.379057 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-9c8e9353-efd2-4294-b50a-21f1246e2e4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed2d7bfba97c4dd7a5280e4cc0b129a8'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:27:33.379636 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-9c8e9353-efd2-4294-b50a-21f1246e2e4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed2d7bfba97c4dd7a5280e4cc0b129a8'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:27:33.380226 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-9c8e9353-efd2-4294-b50a-21f1246e2e4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed2d7bfba97c4dd7a5280e4cc0b129a8'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:27:33.398904 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1456/2901] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 14:27:33 2022] DELETE /identity/v3/users/ed2d7bfba97c4dd7a5280e4cc0b129a8 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:27:33.676817 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93e5f90-9c5f-43c2-9b27-1171dc446f4c admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:27:33.678472 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-e93e5f90-9c5f-43c2-9b27-1171dc446f4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:27:33.693581 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93e5f90-9c5f-43c2-9b27-1171dc446f4c admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:27:33.751541 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae8e0ff-7437-4ec7-a801-989dc2825ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:27:33.752164 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae8e0ff-7437-4ec7-a801-989dc2825ca5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:27:33.752446 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae8e0ff-7437-4ec7-a801-989dc2825ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:27:33.752724 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae8e0ff-7437-4ec7-a801-989dc2825ca5 admin admin] PATH_INFO: `/v3/projects/2a2ff5bb83d24a39939cb063159b2b68` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:27:33.753947 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae8e0ff-7437-4ec7-a801-989dc2825ca5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a2ff5bb83d24a39939cb063159b2b68)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:27:33.756411 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae8e0ff-7437-4ec7-a801-989dc2825ca5 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:27:33.806967 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-1ae8e0ff-7437-4ec7-a801-989dc2825ca5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a2ff5bb83d24a39939cb063159b2b68', 'request_id': 'req-1ae8e0ff-7437-4ec7-a801-989dc2825ca5'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:27:33.813878 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-1ae8e0ff-7437-4ec7-a801-989dc2825ca5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a2ff5bb83d24a39939cb063159b2b68', 'request_id': 'req-1ae8e0ff-7437-4ec7-a801-989dc2825ca5'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:27:33.819315 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-1ae8e0ff-7437-4ec7-a801-989dc2825ca5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a2ff5bb83d24a39939cb063159b2b68', 'request_id': 'req-1ae8e0ff-7437-4ec7-a801-989dc2825ca5'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:27:33.831508 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-1ae8e0ff-7437-4ec7-a801-989dc2825ca5 admin admin] The token cache is being invalidate because project 2a2ff5bb83d24a39939cb063159b2b68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:27:33.831795 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-1ae8e0ff-7437-4ec7-a801-989dc2825ca5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:27:33.832448 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-1ae8e0ff-7437-4ec7-a801-989dc2825ca5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:27:33.834008 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1446/2902] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 14:27:33 2022] DELETE /identity/v3/projects/2a2ff5bb83d24a39939cb063159b2b68 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:27:33.838134 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8e9353-efd2-4294-b50a-21f1246e2e4e admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:27:33.839606 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-9c8e9353-efd2-4294-b50a-21f1246e2e4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:27:33.852126 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8e9353-efd2-4294-b50a-21f1246e2e4e admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:27:33.909957 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15faf068-b1be-4f78-bf8b-8b5150aa41b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:27:33.910547 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15faf068-b1be-4f78-bf8b-8b5150aa41b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:27:33.910828 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15faf068-b1be-4f78-bf8b-8b5150aa41b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:27:33.911100 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15faf068-b1be-4f78-bf8b-8b5150aa41b7 admin admin] PATH_INFO: `/v3/users/a0893c93f4b0444bb11c5d13b28fcca9` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:27:33.912241 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15faf068-b1be-4f78-bf8b-8b5150aa41b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0893c93f4b0444bb11c5d13b28fcca9)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:27:33.914181 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15faf068-b1be-4f78-bf8b-8b5150aa41b7 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:27:33.959941 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-15faf068-b1be-4f78-bf8b-8b5150aa41b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0893c93f4b0444bb11c5d13b28fcca9'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:27:33.969630 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-15faf068-b1be-4f78-bf8b-8b5150aa41b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0893c93f4b0444bb11c5d13b28fcca9'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:27:33.981129 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-15faf068-b1be-4f78-bf8b-8b5150aa41b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0893c93f4b0444bb11c5d13b28fcca9'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:27:33.982135 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-15faf068-b1be-4f78-bf8b-8b5150aa41b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0893c93f4b0444bb11c5d13b28fcca9'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:27:33.983090 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-15faf068-b1be-4f78-bf8b-8b5150aa41b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0893c93f4b0444bb11c5d13b28fcca9'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:27:33.992003 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:27:34.006598 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1457/2903] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 14:27:33 2022] DELETE /identity/v3/users/a0893c93f4b0444bb11c5d13b28fcca9 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:27:34.277742 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae8e0ff-7437-4ec7-a801-989dc2825ca5 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:27:34.280200 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-1ae8e0ff-7437-4ec7-a801-989dc2825ca5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:27:34.298351 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae8e0ff-7437-4ec7-a801-989dc2825ca5 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:27:34.347883 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:27:34.358933 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1487d785-0ff8-49a7-8b64-44380ac51d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:27:34.360073 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1487d785-0ff8-49a7-8b64-44380ac51d7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:27:34.360352 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1487d785-0ff8-49a7-8b64-44380ac51d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:27:34.360628 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1487d785-0ff8-49a7-8b64-44380ac51d7f admin admin] PATH_INFO: `/v3/projects/88535297250b4ec698234db72ab168b9` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:27:34.361799 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1487d785-0ff8-49a7-8b64-44380ac51d7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=88535297250b4ec698234db72ab168b9)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:27:34.363720 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1487d785-0ff8-49a7-8b64-44380ac51d7f admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:27:34.411757 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-1487d785-0ff8-49a7-8b64-44380ac51d7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88535297250b4ec698234db72ab168b9', 'request_id': 'req-1487d785-0ff8-49a7-8b64-44380ac51d7f'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:27:34.418385 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-1487d785-0ff8-49a7-8b64-44380ac51d7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88535297250b4ec698234db72ab168b9', 'request_id': 'req-1487d785-0ff8-49a7-8b64-44380ac51d7f'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:27:34.424740 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-1487d785-0ff8-49a7-8b64-44380ac51d7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88535297250b4ec698234db72ab168b9', 'request_id': 'req-1487d785-0ff8-49a7-8b64-44380ac51d7f'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:27:34.439132 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-1487d785-0ff8-49a7-8b64-44380ac51d7f admin admin] The token cache is being invalidate because project 88535297250b4ec698234db72ab168b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:27:34.439428 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-1487d785-0ff8-49a7-8b64-44380ac51d7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:27:34.440097 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-1487d785-0ff8-49a7-8b64-44380ac51d7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:27:34.441812 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1447/2904] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 14:27:34 2022] DELETE /identity/v3/projects/88535297250b4ec698234db72ab168b9 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:27:34.447108 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c199b86-5b4e-459e-8c9c-60072c0c5bfd None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:27:34.447304 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c199b86-5b4e-459e-8c9c-60072c0c5bfd None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:27:34.447570 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c199b86-5b4e-459e-8c9c-60072c0c5bfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:27:34.471721 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-7c199b86-5b4e-459e-8c9c-60072c0c5bfd None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:27:34.473227 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-7c199b86-5b4e-459e-8c9c-60072c0c5bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:27:34.539625 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1458/2905] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:27:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:27:34.548037 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7742e8f-5bbe-45b0-abca-7e458b715f39 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:27:34.548320 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7742e8f-5bbe-45b0-abca-7e458b715f39 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:27:34.548590 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7742e8f-5bbe-45b0-abca-7e458b715f39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:27:34.571139 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-b7742e8f-5bbe-45b0-abca-7e458b715f39 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:27:34.572629 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-b7742e8f-5bbe-45b0-abca-7e458b715f39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:27:34.581836 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1448/2906] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:27:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:27:34.586532 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c199b86-5b4e-459e-8c9c-60072c0c5bfd None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:27:34.597099 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c199b86-5b4e-459e-8c9c-60072c0c5bfd None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:27:34.601864 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38675803-ae7e-4b29-9fbb-c0f376a07763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:27:34.602492 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38675803-ae7e-4b29-9fbb-c0f376a07763 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:27:34.602777 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38675803-ae7e-4b29-9fbb-c0f376a07763 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:27:34.603062 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38675803-ae7e-4b29-9fbb-c0f376a07763 admin admin] PATH_INFO: `/v3/domains` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:27:34.603891 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38675803-ae7e-4b29-9fbb-c0f376a07763 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:27:34.604205 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38675803-ae7e-4b29-9fbb-c0f376a07763 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:27:34.605873 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38675803-ae7e-4b29-9fbb-c0f376a07763 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:27:34.618630 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1459/2907] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Jul 26 14:27:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:27:34.623283 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7742e8f-5bbe-45b0-abca-7e458b715f39 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:27:34.633784 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7742e8f-5bbe-45b0-abca-7e458b715f39 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:27:34.640239 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8507ada0-2faf-47de-9904-405e7e63d35b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:27:34.640878 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8507ada0-2faf-47de-9904-405e7e63d35b admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:27:34.641167 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8507ada0-2faf-47de-9904-405e7e63d35b admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:27:34.641445 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8507ada0-2faf-47de-9904-405e7e63d35b admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:27:34.642163 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8507ada0-2faf-47de-9904-405e7e63d35b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:27:34.644130 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8507ada0-2faf-47de-9904-405e7e63d35b admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:27:34.679830 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1449/2908] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 14:27:34 2022] POST /identity/v3/projects => generated 367 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:27:34.684682 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38675803-ae7e-4b29-9fbb-c0f376a07763 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:27:34.695260 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38675803-ae7e-4b29-9fbb-c0f376a07763 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:27:34.767560 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903b6826-9adc-450f-a285-f535d1f5ab84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:27:34.768096 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903b6826-9adc-450f-a285-f535d1f5ab84 admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:27:34.768382 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903b6826-9adc-450f-a285-f535d1f5ab84 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:27:34.768662 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903b6826-9adc-450f-a285-f535d1f5ab84 admin admin] PATH_INFO: `/v3/users` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:27:34.769331 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903b6826-9adc-450f-a285-f535d1f5ab84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:27:34.771271 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903b6826-9adc-450f-a285-f535d1f5ab84 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:27:34.793639 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1460/2909] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 14:27:34 2022] POST /identity/v3/users => generated 284 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:27:34.799296 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8507ada0-2faf-47de-9904-405e7e63d35b admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:27:34.811161 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8507ada0-2faf-47de-9904-405e7e63d35b admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:27:34.815792 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0725bbd-9078-4b01-a30b-955dfcdfdaf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:27:34.816397 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0725bbd-9078-4b01-a30b-955dfcdfdaf1 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:27:34.816677 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0725bbd-9078-4b01-a30b-955dfcdfdaf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:27:34.816952 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0725bbd-9078-4b01-a30b-955dfcdfdaf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:27:34.817696 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0725bbd-9078-4b01-a30b-955dfcdfdaf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:27:34.819293 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0725bbd-9078-4b01-a30b-955dfcdfdaf1 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:27:34.829182 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1450/2910] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:27:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:27:34.833294 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903b6826-9adc-450f-a285-f535d1f5ab84 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:27:34.843291 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903b6826-9adc-450f-a285-f535d1f5ab84 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:27:34.848680 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0c93ae-2099-4601-9349-76764d3fe613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:27:34.849295 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0c93ae-2099-4601-9349-76764d3fe613 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:27:34.849609 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0c93ae-2099-4601-9349-76764d3fe613 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:27:34.849885 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0c93ae-2099-4601-9349-76764d3fe613 admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:27:34.850580 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0c93ae-2099-4601-9349-76764d3fe613 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:27:34.852164 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0c93ae-2099-4601-9349-76764d3fe613 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:27:34.863637 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1461/2911] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:27:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:27:34.867900 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0725bbd-9078-4b01-a30b-955dfcdfdaf1 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:27:34.877892 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0725bbd-9078-4b01-a30b-955dfcdfdaf1 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:27:34.882972 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e2f7fc-71ac-4b68-abab-d64e470606fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:27:34.883629 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e2f7fc-71ac-4b68-abab-d64e470606fb admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:27:34.883909 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e2f7fc-71ac-4b68-abab-d64e470606fb admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:27:34.884181 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e2f7fc-71ac-4b68-abab-d64e470606fb admin admin] PATH_INFO: `/v3/projects/d4c5a8eca82449bfaccf1e5d407e3cbe/users/8c1cc812b732405db4b5d0badf895f16/roles/759f389b221049c4b01faf3b984cd8d7` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:27:34.898668 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e2f7fc-71ac-4b68-abab-d64e470606fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4c5a8eca82449bfaccf1e5d407e3cbe, user_id=8c1cc812b732405db4b5d0badf895f16, role_id=759f389b221049c4b01faf3b984cd8d7)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:27:34.901280 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e2f7fc-71ac-4b68-abab-d64e470606fb admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:27:34.914686 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1451/2912] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 14:27:34 2022] PUT /identity/v3/projects/d4c5a8eca82449bfaccf1e5d407e3cbe/users/8c1cc812b732405db4b5d0badf895f16/roles/759f389b221049c4b01faf3b984cd8d7 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:27:34.926189 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a3aa43-cfd9-40ec-9996-bed8350bfe66 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:27:34.926479 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a3aa43-cfd9-40ec-9996-bed8350bfe66 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:27:34.926760 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a3aa43-cfd9-40ec-9996-bed8350bfe66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:27:34.953522 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-d2a3aa43-cfd9-40ec-9996-bed8350bfe66 None None] MFA Rules not processed for user `8c1cc812b732405db4b5d0badf895f16`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:27:34.954858 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-d2a3aa43-cfd9-40ec-9996-bed8350bfe66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:27:34.999448 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:27:35.023634 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1462/2913] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:27:34 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:27:35.036048 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e2f7fc-71ac-4b68-abab-d64e470606fb admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:27:35.037898 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-a3e2f7fc-71ac-4b68-abab-d64e470606fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:27:35.095477 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e2f7fc-71ac-4b68-abab-d64e470606fb admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:27:35.099262 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b9ef6a-5755-4552-af4a-43c1fdf15cc7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eaf065aef9a046b2adaec34816a336b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:27:35.099879 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b9ef6a-5755-4552-af4a-43c1fdf15cc7 service glance] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:27:35.100163 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b9ef6a-5755-4552-af4a-43c1fdf15cc7 service glance] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:27:35.100441 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b9ef6a-5755-4552-af4a-43c1fdf15cc7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:27:35.108661 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b9ef6a-5755-4552-af4a-43c1fdf15cc7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:27:35.110259 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b9ef6a-5755-4552-af4a-43c1fdf15cc7 service glance] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:27:35.118941 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1452/2914] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:27:35 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:27:45.020496 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=8 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:27:45.115021 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=81,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:27:55.243108 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a3aa43-cfd9-40ec-9996-bed8350bfe66 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:27:55.244738 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-d2a3aa43-cfd9-40ec-9996-bed8350bfe66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:27:55.309163 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a3aa43-cfd9-40ec-9996-bed8350bfe66 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:27:55.313409 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed46ad1b-e70c-4a02-8da4-f0faeef78a5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:27:55.314067 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed46ad1b-e70c-4a02-8da4-f0faeef78a5e service cinder] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:27:55.314346 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed46ad1b-e70c-4a02-8da4-f0faeef78a5e service cinder] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:27:55.314619 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed46ad1b-e70c-4a02-8da4-f0faeef78a5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:27:55.316392 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-ed46ad1b-e70c-4a02-8da4-f0faeef78a5e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:27:55.379161 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed46ad1b-e70c-4a02-8da4-f0faeef78a5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:27:55.380860 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed46ad1b-e70c-4a02-8da4-f0faeef78a5e service cinder] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:27:55.390782 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1463/2915] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:27:55 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:28:05.386166 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=48 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:28:08.715357 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab50e858-61fc-4b1e-8d01-e52c481ff2c7 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:28:08.715903 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab50e858-61fc-4b1e-8d01-e52c481ff2c7 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:28:08.715903 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab50e858-61fc-4b1e-8d01-e52c481ff2c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:28:08.734918 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-ab50e858-61fc-4b1e-8d01-e52c481ff2c7 None None] MFA Rules not processed for user `7745969be55b487dbb15b216a3016731`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:28:08.736142 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-ab50e858-61fc-4b1e-8d01-e52c481ff2c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:28:08.743310 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1453/2916] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:28:08 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:28:08.785671 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed46ad1b-e70c-4a02-8da4-f0faeef78a5e service cinder] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:28:08.794753 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed46ad1b-e70c-4a02-8da4-f0faeef78a5e service cinder] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:28:08.798956 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4895645-3015-457f-a225-30f2207d7c46 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:28:08.799585 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4895645-3015-457f-a225-30f2207d7c46 service cinder] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:28:08.799893 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4895645-3015-457f-a225-30f2207d7c46 service cinder] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:28:08.800165 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4895645-3015-457f-a225-30f2207d7c46 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:28:08.801929 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-a4895645-3015-457f-a225-30f2207d7c46 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:28:08.877925 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4895645-3015-457f-a225-30f2207d7c46 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:28:08.879268 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4895645-3015-457f-a225-30f2207d7c46 service cinder] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:28:08.908488 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1464/2917] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:28:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:28:09.217843 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab50e858-61fc-4b1e-8d01-e52c481ff2c7 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:28:09.219828 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-ab50e858-61fc-4b1e-8d01-e52c481ff2c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:28:09.284220 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab50e858-61fc-4b1e-8d01-e52c481ff2c7 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:28:09.289786 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1342fe08-4068-43ef-87ec-48e97e6fb3ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:28:09.290681 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1342fe08-4068-43ef-87ec-48e97e6fb3ed service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:28:09.291109 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1342fe08-4068-43ef-87ec-48e97e6fb3ed service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:28:09.291533 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1342fe08-4068-43ef-87ec-48e97e6fb3ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:28:09.294413 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-1342fe08-4068-43ef-87ec-48e97e6fb3ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:28:09.312800 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1342fe08-4068-43ef-87ec-48e97e6fb3ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:28:09.314327 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1342fe08-4068-43ef-87ec-48e97e6fb3ed service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:28:09.327673 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1454/2918] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:28:09 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:28:09.838484 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4895645-3015-457f-a225-30f2207d7c46 service cinder] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:28:09.839907 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-a4895645-3015-457f-a225-30f2207d7c46 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:28:09.923262 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4895645-3015-457f-a225-30f2207d7c46 service cinder] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:28:09.928264 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e880fd48-87b6-4610-b78e-e6b5e6a84578 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '835cbc13756b41b598142df3e35f5492', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:28:09.928887 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e880fd48-87b6-4610-b78e-e6b5e6a84578 service placement] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:28:09.929168 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e880fd48-87b6-4610-b78e-e6b5e6a84578 service placement] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:28:09.929437 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e880fd48-87b6-4610-b78e-e6b5e6a84578 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:28:09.931683 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-e880fd48-87b6-4610-b78e-e6b5e6a84578 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:28:09.944240 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e880fd48-87b6-4610-b78e-e6b5e6a84578 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:28:09.947424 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e880fd48-87b6-4610-b78e-e6b5e6a84578 service placement] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:28:09.957380 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1465/2919] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:28:09 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:28:15.565811 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1342fe08-4068-43ef-87ec-48e97e6fb3ed service neutron] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:28:15.569028 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-1342fe08-4068-43ef-87ec-48e97e6fb3ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:28:15.588124 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1342fe08-4068-43ef-87ec-48e97e6fb3ed service neutron] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:28:15.594807 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82016e3-31d3-48cd-887a-b7b9610f2617 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c69161cd520b4b29bf50f3bf2f1c033b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:28:15.595461 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82016e3-31d3-48cd-887a-b7b9610f2617 service nova] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:28:15.595740 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82016e3-31d3-48cd-887a-b7b9610f2617 service nova] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:28:15.596014 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82016e3-31d3-48cd-887a-b7b9610f2617 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:28:15.597873 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-d82016e3-31d3-48cd-887a-b7b9610f2617 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:28:15.611824 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82016e3-31d3-48cd-887a-b7b9610f2617 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:28:15.613587 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82016e3-31d3-48cd-887a-b7b9610f2617 service nova] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:28:15.627331 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1455/2920] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:28:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:28:19.952244 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=70 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:28:20.461909 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e880fd48-87b6-4610-b78e-e6b5e6a84578 service placement] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:28:20.527403 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e880fd48-87b6-4610-b78e-e6b5e6a84578 service placement] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:28:20.532312 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e99000-123a-47d0-981d-33e2b2e601ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:28:20.532962 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e99000-123a-47d0-981d-33e2b2e601ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:28:20.533243 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e99000-123a-47d0-981d-33e2b2e601ba admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:28:20.533558 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e99000-123a-47d0-981d-33e2b2e601ba admin admin] PATH_INFO: `/v3/users/8c1cc812b732405db4b5d0badf895f16` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:28:20.534699 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e99000-123a-47d0-981d-33e2b2e601ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c1cc812b732405db4b5d0badf895f16)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:28:20.536637 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e99000-123a-47d0-981d-33e2b2e601ba admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:28:20.586141 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-75e99000-123a-47d0-981d-33e2b2e601ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c1cc812b732405db4b5d0badf895f16'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:28:20.594702 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-75e99000-123a-47d0-981d-33e2b2e601ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c1cc812b732405db4b5d0badf895f16'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:28:20.602617 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-75e99000-123a-47d0-981d-33e2b2e601ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c1cc812b732405db4b5d0badf895f16'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:28:20.603322 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-75e99000-123a-47d0-981d-33e2b2e601ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c1cc812b732405db4b5d0badf895f16'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:28:20.603948 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-75e99000-123a-47d0-981d-33e2b2e601ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c1cc812b732405db4b5d0badf895f16'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:28:20.623259 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1466/2921] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 14:28:20 2022] DELETE /identity/v3/users/8c1cc812b732405db4b5d0badf895f16 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:28:20.628501 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213ee316-56ad-492a-9836-3b96183041dd None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:28:20.629027 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213ee316-56ad-492a-9836-3b96183041dd None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:28:20.629122 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213ee316-56ad-492a-9836-3b96183041dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:28:20.660921 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-213ee316-56ad-492a-9836-3b96183041dd None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:28:20.662983 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-213ee316-56ad-492a-9836-3b96183041dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:28:20.737582 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1456/2922] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:28:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:28:20.747606 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e99000-123a-47d0-981d-33e2b2e601ba admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:28:20.749142 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-75e99000-123a-47d0-981d-33e2b2e601ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:28:20.817719 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e99000-123a-47d0-981d-33e2b2e601ba admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:28:20.821548 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218f6f35-b39d-4878-8db3-a11d57c0132c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:28:20.822184 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218f6f35-b39d-4878-8db3-a11d57c0132c service neutron] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:28:20.822468 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218f6f35-b39d-4878-8db3-a11d57c0132c service neutron] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:28:20.822746 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218f6f35-b39d-4878-8db3-a11d57c0132c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:28:20.833531 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218f6f35-b39d-4878-8db3-a11d57c0132c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:28:20.834994 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218f6f35-b39d-4878-8db3-a11d57c0132c service neutron] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:28:20.845671 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1467/2923] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:28:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:28:21.141941 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213ee316-56ad-492a-9836-3b96183041dd None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:28:21.143519 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-213ee316-56ad-492a-9836-3b96183041dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:28:21.156490 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213ee316-56ad-492a-9836-3b96183041dd None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:28:21.161340 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f28fd4c-4670-4cc6-806d-f431a52623a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:28:21.162163 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f28fd4c-4670-4cc6-806d-f431a52623a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:28:21.162441 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f28fd4c-4670-4cc6-806d-f431a52623a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:28:21.162713 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f28fd4c-4670-4cc6-806d-f431a52623a4 admin admin] PATH_INFO: `/v3/projects/d4c5a8eca82449bfaccf1e5d407e3cbe` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:28:21.163881 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f28fd4c-4670-4cc6-806d-f431a52623a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4c5a8eca82449bfaccf1e5d407e3cbe)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:28:21.165821 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f28fd4c-4670-4cc6-806d-f431a52623a4 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:28:21.221437 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-4f28fd4c-4670-4cc6-806d-f431a52623a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4c5a8eca82449bfaccf1e5d407e3cbe', 'request_id': 'req-4f28fd4c-4670-4cc6-806d-f431a52623a4'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:28:21.228480 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-4f28fd4c-4670-4cc6-806d-f431a52623a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4c5a8eca82449bfaccf1e5d407e3cbe', 'request_id': 'req-4f28fd4c-4670-4cc6-806d-f431a52623a4'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:28:21.234597 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-4f28fd4c-4670-4cc6-806d-f431a52623a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4c5a8eca82449bfaccf1e5d407e3cbe', 'request_id': 'req-4f28fd4c-4670-4cc6-806d-f431a52623a4'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:28:21.248429 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-4f28fd4c-4670-4cc6-806d-f431a52623a4 admin admin] The token cache is being invalidate because project d4c5a8eca82449bfaccf1e5d407e3cbe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:28:21.248720 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-4f28fd4c-4670-4cc6-806d-f431a52623a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:28:21.249691 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-4f28fd4c-4670-4cc6-806d-f431a52623a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:28:21.251222 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1457/2924] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 14:28:21 2022] DELETE /identity/v3/projects/d4c5a8eca82449bfaccf1e5d407e3cbe => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:28:21.256415 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d88c59-1e52-411c-81e3-51b644625e52 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:28:21.257230 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d88c59-1e52-411c-81e3-51b644625e52 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:28:21.257542 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d88c59-1e52-411c-81e3-51b644625e52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:28:21.277668 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-85d88c59-1e52-411c-81e3-51b644625e52 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:28:21.278923 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-85d88c59-1e52-411c-81e3-51b644625e52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:28:21.299163 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:28:21.350723 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1468/2925] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:28:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:28:21.360163 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed3dca1-fc21-49c8-aae2-30d1bdd135c1 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:28:21.360441 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed3dca1-fc21-49c8-aae2-30d1bdd135c1 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:28:21.360714 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed3dca1-fc21-49c8-aae2-30d1bdd135c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:28:21.375893 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:28:21.382411 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-6ed3dca1-fc21-49c8-aae2-30d1bdd135c1 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:28:21.384647 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-6ed3dca1-fc21-49c8-aae2-30d1bdd135c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:28:21.397031 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1458/2926] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:28:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:28:21.401836 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d88c59-1e52-411c-81e3-51b644625e52 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:28:21.418038 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d88c59-1e52-411c-81e3-51b644625e52 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:28:21.424083 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1befdc74-1168-45a9-a793-9832c865cbe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:28:21.424807 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1befdc74-1168-45a9-a793-9832c865cbe2 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:28:21.425376 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1befdc74-1168-45a9-a793-9832c865cbe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:28:21.425565 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1befdc74-1168-45a9-a793-9832c865cbe2 admin admin] PATH_INFO: `/v3/domains` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:28:21.426324 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1befdc74-1168-45a9-a793-9832c865cbe2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86933) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:28:21.426717 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1befdc74-1168-45a9-a793-9832c865cbe2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:28:21.428413 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1befdc74-1168-45a9-a793-9832c865cbe2 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:28:21.447605 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1469/2927] 10.222.0.58 () {58 vars in 1153 bytes} [Tue Jul 26 14:28:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:28:21.452153 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed3dca1-fc21-49c8-aae2-30d1bdd135c1 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:28:21.463850 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed3dca1-fc21-49c8-aae2-30d1bdd135c1 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:28:21.470081 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c46d0c4-091f-4e7a-9094-f7641f616d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:28:21.471015 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c46d0c4-091f-4e7a-9094-f7641f616d2c admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:28:21.471859 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c46d0c4-091f-4e7a-9094-f7641f616d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:28:21.472289 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c46d0c4-091f-4e7a-9094-f7641f616d2c admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:28:21.473348 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c46d0c4-091f-4e7a-9094-f7641f616d2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:28:21.476430 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c46d0c4-091f-4e7a-9094-f7641f616d2c admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:28:21.509663 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1459/2928] 10.222.0.58 () {60 vars in 1153 bytes} [Tue Jul 26 14:28:21 2022] POST /identity/v3/projects => generated 373 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:28:21.516151 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1befdc74-1168-45a9-a793-9832c865cbe2 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:28:21.527440 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1befdc74-1168-45a9-a793-9832c865cbe2 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:28:21.593006 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76092de-fe85-4404-9944-c4862fbe4b52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:28:21.593825 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76092de-fe85-4404-9944-c4862fbe4b52 admin admin] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:28:21.594107 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76092de-fe85-4404-9944-c4862fbe4b52 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:28:21.594381 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76092de-fe85-4404-9944-c4862fbe4b52 admin admin] PATH_INFO: `/v3/users` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:28:21.595056 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76092de-fe85-4404-9944-c4862fbe4b52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:28:21.596971 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76092de-fe85-4404-9944-c4862fbe4b52 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:28:21.625837 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1470/2929] 10.222.0.58 () {60 vars in 1144 bytes} [Tue Jul 26 14:28:21 2022] POST /identity/v3/users => generated 287 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 14:28:21.630412 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c46d0c4-091f-4e7a-9094-f7641f616d2c admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:28:21.642031 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c46d0c4-091f-4e7a-9094-f7641f616d2c admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:28:21.647461 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f130313-be4e-433d-9feb-3d030eda2a4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:28:21.648136 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f130313-be4e-433d-9feb-3d030eda2a4f admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:28:21.648422 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f130313-be4e-433d-9feb-3d030eda2a4f admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:28:21.648699 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f130313-be4e-433d-9feb-3d030eda2a4f admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:28:21.649405 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f130313-be4e-433d-9feb-3d030eda2a4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:28:21.651043 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f130313-be4e-433d-9feb-3d030eda2a4f admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:28:21.661889 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1460/2930] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:28:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:28:21.667671 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76092de-fe85-4404-9944-c4862fbe4b52 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:28:21.680193 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76092de-fe85-4404-9944-c4862fbe4b52 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:28:21.686534 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442786b8-4d90-47d7-979a-f442ca249ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:28:21.687831 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442786b8-4d90-47d7-979a-f442ca249ad8 admin admin] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:28:21.688223 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442786b8-4d90-47d7-979a-f442ca249ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:28:21.688549 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442786b8-4d90-47d7-979a-f442ca249ad8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:28:21.689315 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442786b8-4d90-47d7-979a-f442ca249ad8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:28:21.691423 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442786b8-4d90-47d7-979a-f442ca249ad8 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:28:21.705695 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1471/2931] 10.222.0.58 () {58 vars in 1122 bytes} [Tue Jul 26 14:28:21 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:28:21.710092 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f130313-be4e-433d-9feb-3d030eda2a4f admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:28:21.720628 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f130313-be4e-433d-9feb-3d030eda2a4f admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:28:21.725385 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d94711-09ce-48ba-b68f-370bbb3bd4e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:28:21.726878 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d94711-09ce-48ba-b68f-370bbb3bd4e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:28:21.726878 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d94711-09ce-48ba-b68f-370bbb3bd4e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:28:21.726878 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d94711-09ce-48ba-b68f-370bbb3bd4e1 admin admin] PATH_INFO: `/v3/projects/6c6db633164d4b44bd19ab9f47d850d9/users/84a8a065fddc47b080acb2cad1045919/roles/759f389b221049c4b01faf3b984cd8d7` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:28:21.750735 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d94711-09ce-48ba-b68f-370bbb3bd4e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c6db633164d4b44bd19ab9f47d850d9, user_id=84a8a065fddc47b080acb2cad1045919, role_id=759f389b221049c4b01faf3b984cd8d7)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:28:21.753451 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d94711-09ce-48ba-b68f-370bbb3bd4e1 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:28:21.768129 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1461/2932] 10.222.0.58 () {60 vars in 1483 bytes} [Tue Jul 26 14:28:21 2022] PUT /identity/v3/projects/6c6db633164d4b44bd19ab9f47d850d9/users/84a8a065fddc47b080acb2cad1045919/roles/759f389b221049c4b01faf3b984cd8d7 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:28:21.778948 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4194e77e-5db0-4b50-a64b-cac4240cd163 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:28:21.779240 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4194e77e-5db0-4b50-a64b-cac4240cd163 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:28:21.779514 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4194e77e-5db0-4b50-a64b-cac4240cd163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:28:21.824149 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-4194e77e-5db0-4b50-a64b-cac4240cd163 None None] MFA Rules not processed for user `84a8a065fddc47b080acb2cad1045919`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:28:21.825386 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-4194e77e-5db0-4b50-a64b-cac4240cd163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:28:21.891477 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1472/2933] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:28:21 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:28:21.902230 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d94711-09ce-48ba-b68f-370bbb3bd4e1 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:28:21.903908 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-d5d94711-09ce-48ba-b68f-370bbb3bd4e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:28:21.961017 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d94711-09ce-48ba-b68f-370bbb3bd4e1 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:28:21.965358 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0b5f9b-5993-4e2c-a485-bf550405c859 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72e400f267fd488d8439426a81ba0a25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:28:21.966015 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0b5f9b-5993-4e2c-a485-bf550405c859 service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:28:21.966302 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0b5f9b-5993-4e2c-a485-bf550405c859 service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:28:21.966583 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0b5f9b-5993-4e2c-a485-bf550405c859 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:28:21.975130 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0b5f9b-5993-4e2c-a485-bf550405c859 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:28:21.976554 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0b5f9b-5993-4e2c-a485-bf550405c859 service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:28:21.985840 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1462/2934] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:28:21 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:28:25.651758 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4194e77e-5db0-4b50-a64b-cac4240cd163 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:28:25.654886 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-4194e77e-5db0-4b50-a64b-cac4240cd163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:28:25.687160 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:28:25.753811 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4194e77e-5db0-4b50-a64b-cac4240cd163 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:28:25.759645 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33efb906-d4b5-41ea-babd-5e3589d13725 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c69161cd520b4b29bf50f3bf2f1c033b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:28:25.760283 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33efb906-d4b5-41ea-babd-5e3589d13725 service nova] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:28:25.760560 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33efb906-d4b5-41ea-babd-5e3589d13725 service nova] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:28:25.760836 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33efb906-d4b5-41ea-babd-5e3589d13725 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:28:25.762720 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-33efb906-d4b5-41ea-babd-5e3589d13725 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:28:25.776710 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33efb906-d4b5-41ea-babd-5e3589d13725 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:28:25.778819 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33efb906-d4b5-41ea-babd-5e3589d13725 service nova] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:28:25.793532 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1473/2935] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:28:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:28:28.913337 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0b5f9b-5993-4e2c-a485-bf550405c859 service cinder] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:28:28.914941 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-6a0b5f9b-5993-4e2c-a485-bf550405c859 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:28:28.985702 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0b5f9b-5993-4e2c-a485-bf550405c859 service cinder] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:28:28.989878 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4750f4ea-6290-4959-a152-99a57acc85f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:28:28.990531 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4750f4ea-6290-4959-a152-99a57acc85f0 service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:28:28.990814 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4750f4ea-6290-4959-a152-99a57acc85f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:28:28.991090 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4750f4ea-6290-4959-a152-99a57acc85f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:28:28.992991 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-4750f4ea-6290-4959-a152-99a57acc85f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:28:29.072373 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:28:29.076806 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4750f4ea-6290-4959-a152-99a57acc85f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:28:29.078958 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4750f4ea-6290-4959-a152-99a57acc85f0 service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:28:29.091473 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1463/2936] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:28:28 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:28:31.148397 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33efb906-d4b5-41ea-babd-5e3589d13725 service nova] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:28:31.161943 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33efb906-d4b5-41ea-babd-5e3589d13725 service nova] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:28:31.168027 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e429fcab-724e-4e06-a6c8-6a71270fdb0a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c69161cd520b4b29bf50f3bf2f1c033b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:28:31.169330 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e429fcab-724e-4e06-a6c8-6a71270fdb0a service nova] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:28:31.169330 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e429fcab-724e-4e06-a6c8-6a71270fdb0a service nova] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:28:31.169330 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e429fcab-724e-4e06-a6c8-6a71270fdb0a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:28:31.171823 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-e429fcab-724e-4e06-a6c8-6a71270fdb0a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:28:31.188233 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e429fcab-724e-4e06-a6c8-6a71270fdb0a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:28:31.189963 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e429fcab-724e-4e06-a6c8-6a71270fdb0a service nova] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:28:31.202298 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1474/2937] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:28:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:28:32.753175 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4750f4ea-6290-4959-a152-99a57acc85f0 service neutron] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:28:32.817822 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4750f4ea-6290-4959-a152-99a57acc85f0 service neutron] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:28:32.822649 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16634b3e-a8b9-434b-bd39-9fa0b6304617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:28:32.823316 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16634b3e-a8b9-434b-bd39-9fa0b6304617 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:28:32.823608 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16634b3e-a8b9-434b-bd39-9fa0b6304617 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:28:32.823887 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16634b3e-a8b9-434b-bd39-9fa0b6304617 admin admin] PATH_INFO: `/v3/users/84a8a065fddc47b080acb2cad1045919` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:28:32.825175 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16634b3e-a8b9-434b-bd39-9fa0b6304617 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84a8a065fddc47b080acb2cad1045919)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:28:32.827688 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16634b3e-a8b9-434b-bd39-9fa0b6304617 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:28:32.892374 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-16634b3e-a8b9-434b-bd39-9fa0b6304617 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84a8a065fddc47b080acb2cad1045919'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:28:32.893055 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-16634b3e-a8b9-434b-bd39-9fa0b6304617 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84a8a065fddc47b080acb2cad1045919'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:28:32.907019 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-16634b3e-a8b9-434b-bd39-9fa0b6304617 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84a8a065fddc47b080acb2cad1045919'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:28:32.907811 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-16634b3e-a8b9-434b-bd39-9fa0b6304617 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84a8a065fddc47b080acb2cad1045919'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:28:32.922066 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-16634b3e-a8b9-434b-bd39-9fa0b6304617 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84a8a065fddc47b080acb2cad1045919'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:28:32.938571 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1464/2938] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 14:28:32 2022] DELETE /identity/v3/users/84a8a065fddc47b080acb2cad1045919 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:28:32.943560 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da550489-5dbd-4716-87ee-7bd553564014 None None] REQUEST_METHOD: `POST` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:28:32.944193 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da550489-5dbd-4716-87ee-7bd553564014 None None] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:28:32.944193 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da550489-5dbd-4716-87ee-7bd553564014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:28:32.965612 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.auth.core [None req-da550489-5dbd-4716-87ee-7bd553564014 None None] MFA Rules not processed for user `f1ff1f65366f4d20a56500bb6bc88c54`. Rule list: `[]` (Enabled: `True`). {{(pid=86933) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:28:32.966864 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-da550489-5dbd-4716-87ee-7bd553564014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:28:33.052489 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1475/2939] 10.222.0.58 () {58 vars in 958 bytes} [Tue Jul 26 14:28:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:28:33.063543 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16634b3e-a8b9-434b-bd39-9fa0b6304617 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:28:33.065114 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-16634b3e-a8b9-434b-bd39-9fa0b6304617 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:28:33.137783 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16634b3e-a8b9-434b-bd39-9fa0b6304617 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:28:33.142214 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeadf3b-7a05-4577-ad8c-502eb307a15d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:28:33.142979 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeadf3b-7a05-4577-ad8c-502eb307a15d service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:28:33.143288 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeadf3b-7a05-4577-ad8c-502eb307a15d service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:28:33.143560 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeadf3b-7a05-4577-ad8c-502eb307a15d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:28:33.153896 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beeadf3b-7a05-4577-ad8c-502eb307a15d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:28:33.155384 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beeadf3b-7a05-4577-ad8c-502eb307a15d service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:28:33.166322 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1465/2940] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:28:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:28:33.496449 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da550489-5dbd-4716-87ee-7bd553564014 None None] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:28:33.498747 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-da550489-5dbd-4716-87ee-7bd553564014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:28:33.516778 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da550489-5dbd-4716-87ee-7bd553564014 None None] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:28:33.521773 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7d9579-9e3c-4028-90e9-e44466d746cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:28:33.522383 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7d9579-9e3c-4028-90e9-e44466d746cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:28:33.522668 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7d9579-9e3c-4028-90e9-e44466d746cf admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:28:33.522947 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7d9579-9e3c-4028-90e9-e44466d746cf admin admin] PATH_INFO: `/v3/projects/6c6db633164d4b44bd19ab9f47d850d9` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:28:33.524402 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7d9579-9e3c-4028-90e9-e44466d746cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c6db633164d4b44bd19ab9f47d850d9)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:28:33.526741 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7d9579-9e3c-4028-90e9-e44466d746cf admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:28:33.586545 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-7d7d9579-9e3c-4028-90e9-e44466d746cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c6db633164d4b44bd19ab9f47d850d9', 'request_id': 'req-7d7d9579-9e3c-4028-90e9-e44466d746cf'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:28:33.596201 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-7d7d9579-9e3c-4028-90e9-e44466d746cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c6db633164d4b44bd19ab9f47d850d9', 'request_id': 'req-7d7d9579-9e3c-4028-90e9-e44466d746cf'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:28:33.602544 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-7d7d9579-9e3c-4028-90e9-e44466d746cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c6db633164d4b44bd19ab9f47d850d9', 'request_id': 'req-7d7d9579-9e3c-4028-90e9-e44466d746cf'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:28:33.614877 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-7d7d9579-9e3c-4028-90e9-e44466d746cf admin admin] The token cache is being invalidate because project 6c6db633164d4b44bd19ab9f47d850d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86933) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:28:33.615167 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-7d7d9579-9e3c-4028-90e9-e44466d746cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:28:33.615827 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-7d7d9579-9e3c-4028-90e9-e44466d746cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:28:33.617347 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1476/2941] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 14:28:33 2022] DELETE /identity/v3/projects/6c6db633164d4b44bd19ab9f47d850d9 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:28:43.161238 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=81,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:28:43.611671 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=86,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:23.872995 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeadf3b-7a05-4577-ad8c-502eb307a15d service neutron] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:23.875305 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-beeadf3b-7a05-4577-ad8c-502eb307a15d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:30:23.887557 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeadf3b-7a05-4577-ad8c-502eb307a15d service neutron] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:23.937955 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0914b76e-ffe6-4157-a44a-bb561e38af59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3223a78866d94c2ba66402911c2dfae9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:23.938582 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0914b76e-ffe6-4157-a44a-bb561e38af59 service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:23.938863 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0914b76e-ffe6-4157-a44a-bb561e38af59 service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:23.939134 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0914b76e-ffe6-4157-a44a-bb561e38af59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:23.940893 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-0914b76e-ffe6-4157-a44a-bb561e38af59 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:30:24.006795 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0914b76e-ffe6-4157-a44a-bb561e38af59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:24.008204 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0914b76e-ffe6-4157-a44a-bb561e38af59 service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:24.018173 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1466/2942] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:30:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:30:30.383400 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7d9579-9e3c-4028-90e9-e44466d746cf admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:30.384906 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-7d7d9579-9e3c-4028-90e9-e44466d746cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:30:30.399942 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7d9579-9e3c-4028-90e9-e44466d746cf admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:30.404848 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4135abff-5c13-45f1-a716-097d8e1b5b71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:30.405435 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4135abff-5c13-45f1-a716-097d8e1b5b71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:30.405760 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4135abff-5c13-45f1-a716-097d8e1b5b71 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:30.406034 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4135abff-5c13-45f1-a716-097d8e1b5b71 admin admin] PATH_INFO: `/v3/users/7745969be55b487dbb15b216a3016731` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:30.407211 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4135abff-5c13-45f1-a716-097d8e1b5b71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7745969be55b487dbb15b216a3016731)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:30.409136 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4135abff-5c13-45f1-a716-097d8e1b5b71 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:30.459291 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-4135abff-5c13-45f1-a716-097d8e1b5b71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7745969be55b487dbb15b216a3016731'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:30:30.467817 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-4135abff-5c13-45f1-a716-097d8e1b5b71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7745969be55b487dbb15b216a3016731'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:30:30.475259 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-4135abff-5c13-45f1-a716-097d8e1b5b71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7745969be55b487dbb15b216a3016731'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:30:30.476012 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-4135abff-5c13-45f1-a716-097d8e1b5b71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7745969be55b487dbb15b216a3016731'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:30:30.476692 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-4135abff-5c13-45f1-a716-097d8e1b5b71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7745969be55b487dbb15b216a3016731'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:30:30.494498 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1477/2943] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 14:30:30 2022] DELETE /identity/v3/users/7745969be55b487dbb15b216a3016731 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:30:30.676150 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0914b76e-ffe6-4157-a44a-bb561e38af59 service neutron] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:30.691882 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0914b76e-ffe6-4157-a44a-bb561e38af59 service neutron] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:30.754765 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad64e01-2cfd-423d-bea8-564a16b40cbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:30.755401 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad64e01-2cfd-423d-bea8-564a16b40cbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:30.755686 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad64e01-2cfd-423d-bea8-564a16b40cbb admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:30.755963 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad64e01-2cfd-423d-bea8-564a16b40cbb admin admin] PATH_INFO: `/v3/projects/35c9722406c24d7982e271119af624d7` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:30.757108 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad64e01-2cfd-423d-bea8-564a16b40cbb admin admin] RBAC: Authorizing `identity:delete_project(project_id=35c9722406c24d7982e271119af624d7)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:30.759041 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad64e01-2cfd-423d-bea8-564a16b40cbb admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:30.806033 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-3ad64e01-2cfd-423d-bea8-564a16b40cbb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35c9722406c24d7982e271119af624d7', 'request_id': 'req-3ad64e01-2cfd-423d-bea8-564a16b40cbb'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:30:30.808010 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:30.814010 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-3ad64e01-2cfd-423d-bea8-564a16b40cbb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35c9722406c24d7982e271119af624d7', 'request_id': 'req-3ad64e01-2cfd-423d-bea8-564a16b40cbb'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:30:30.819597 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-3ad64e01-2cfd-423d-bea8-564a16b40cbb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35c9722406c24d7982e271119af624d7', 'request_id': 'req-3ad64e01-2cfd-423d-bea8-564a16b40cbb'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:30:30.831774 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-3ad64e01-2cfd-423d-bea8-564a16b40cbb admin admin] The token cache is being invalidate because project 35c9722406c24d7982e271119af624d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:30:30.832066 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-3ad64e01-2cfd-423d-bea8-564a16b40cbb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:30:30.832693 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-3ad64e01-2cfd-423d-bea8-564a16b40cbb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:30:30.834229 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1467/2944] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 14:30:30 2022] DELETE /identity/v3/projects/35c9722406c24d7982e271119af624d7 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:30:30.838362 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4135abff-5c13-45f1-a716-097d8e1b5b71 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:30.839810 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-4135abff-5c13-45f1-a716-097d8e1b5b71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:30:30.853631 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4135abff-5c13-45f1-a716-097d8e1b5b71 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:30.910973 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9786b6cc-b69e-4e46-ba87-d9a020dc9435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:30.911623 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9786b6cc-b69e-4e46-ba87-d9a020dc9435 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:30.911904 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9786b6cc-b69e-4e46-ba87-d9a020dc9435 admin admin] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:30.912181 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9786b6cc-b69e-4e46-ba87-d9a020dc9435 admin admin] PATH_INFO: `/v3/users/9fd120bb45394b8984de9455dc7d3841` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:30.913342 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9786b6cc-b69e-4e46-ba87-d9a020dc9435 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9fd120bb45394b8984de9455dc7d3841)` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:30.915324 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9786b6cc-b69e-4e46-ba87-d9a020dc9435 admin admin] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:30.959761 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-9786b6cc-b69e-4e46-ba87-d9a020dc9435 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9fd120bb45394b8984de9455dc7d3841'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:30:30.966907 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-9786b6cc-b69e-4e46-ba87-d9a020dc9435 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9fd120bb45394b8984de9455dc7d3841'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:30:30.974379 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-9786b6cc-b69e-4e46-ba87-d9a020dc9435 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9fd120bb45394b8984de9455dc7d3841'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:30:30.975039 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-9786b6cc-b69e-4e46-ba87-d9a020dc9435 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9fd120bb45394b8984de9455dc7d3841'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:30:30.975642 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.notifications [None req-9786b6cc-b69e-4e46-ba87-d9a020dc9435 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9fd120bb45394b8984de9455dc7d3841'} {{(pid=86933) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:30:30.984065 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:30.996141 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1478/2945] 10.222.0.58 () {58 vars in 1224 bytes} [Tue Jul 26 14:30:30 2022] DELETE /identity/v3/users/9fd120bb45394b8984de9455dc7d3841 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:30:31.170004 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad64e01-2cfd-423d-bea8-564a16b40cbb admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:30:31.172208 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-3ad64e01-2cfd-423d-bea8-564a16b40cbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:30:31.188005 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad64e01-2cfd-423d-bea8-564a16b40cbb admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:30:31.246223 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e0f281-9aaa-4339-b310-1d6015ccfb75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1ff1f65366f4d20a56500bb6bc88c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e341f5b1be34081bfdda39547d48a7c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:30:31.246863 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e0f281-9aaa-4339-b310-1d6015ccfb75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:30:31.247141 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e0f281-9aaa-4339-b310-1d6015ccfb75 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:30:31.247413 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e0f281-9aaa-4339-b310-1d6015ccfb75 admin admin] PATH_INFO: `/v3/projects/c87c169bd8dd4c4797c07187e0fecdc4` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:30:31.248695 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e0f281-9aaa-4339-b310-1d6015ccfb75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c87c169bd8dd4c4797c07187e0fecdc4)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:30:31.250620 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e0f281-9aaa-4339-b310-1d6015ccfb75 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:30:31.295790 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-e2e0f281-9aaa-4339-b310-1d6015ccfb75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c87c169bd8dd4c4797c07187e0fecdc4', 'request_id': 'req-e2e0f281-9aaa-4339-b310-1d6015ccfb75'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:30:31.301821 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-e2e0f281-9aaa-4339-b310-1d6015ccfb75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c87c169bd8dd4c4797c07187e0fecdc4', 'request_id': 'req-e2e0f281-9aaa-4339-b310-1d6015ccfb75'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:30:31.307245 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-e2e0f281-9aaa-4339-b310-1d6015ccfb75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c87c169bd8dd4c4797c07187e0fecdc4', 'request_id': 'req-e2e0f281-9aaa-4339-b310-1d6015ccfb75'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:30:31.319175 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-e2e0f281-9aaa-4339-b310-1d6015ccfb75 admin admin] The token cache is being invalidate because project c87c169bd8dd4c4797c07187e0fecdc4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 14:30:31.319465 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-e2e0f281-9aaa-4339-b310-1d6015ccfb75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:30:31.320103 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-e2e0f281-9aaa-4339-b310-1d6015ccfb75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 14:30:31.321641 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1468/2946] 10.222.0.58 () {58 vars in 1233 bytes} [Tue Jul 26 14:30:31 2022] DELETE /identity/v3/projects/c87c169bd8dd4c4797c07187e0fecdc4 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:30:40.999938 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=3,keystone:DELETE=1 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:30:41.316212 dv-cinder-825492-30-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=56,keystone:DELETE=10,keystone:INSERT=2 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:31:07.225546 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9786b6cc-b69e-4e46-ba87-d9a020dc9435 admin admin] Authenticating user token {{(pid=86933) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:31:07.227200 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-9786b6cc-b69e-4e46-ba87-d9a020dc9435 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:31:07.319343 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9786b6cc-b69e-4e46-ba87-d9a020dc9435 admin admin] Validating token access rules against request {{(pid=86933) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:31:07.386506 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64cd0e8-5ce1-4e45-8289-444286fb5c10 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '835cbc13756b41b598142df3e35f5492', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e8bb4385c204210a1bc77b1f0d654f3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86933) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:31:07.387141 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64cd0e8-5ce1-4e45-8289-444286fb5c10 service placement] REQUEST_METHOD: `GET` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:31:07.387421 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64cd0e8-5ce1-4e45-8289-444286fb5c10 service placement] SCRIPT_NAME: `/identity` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:31:07.387695 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64cd0e8-5ce1-4e45-8289-444286fb5c10 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86933) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:31:07.389495 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.fernet_utils [None req-d64cd0e8-5ce1-4e45-8289-444286fb5c10 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86933) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:31:07.401409 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64cd0e8-5ce1-4e45-8289-444286fb5c10 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:31:07.403018 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64cd0e8-5ce1-4e45-8289-444286fb5c10 service placement] RBAC: Authorization granted {{(pid=86933) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:31:07.413849 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: [pid: 86933|app: 0|req: 1479/2947] 10.222.0.58 () {60 vars in 1365 bytes} [Tue Jul 26 14:31:07 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:31:17.409066 dv-cinder-825492-30-iscsi devstack@keystone.service[86933]: DEBUG dbcounter [-] [86933] Writing DB stats keystone:SELECT=58 {{(pid=86933) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}